Re: [AOLSERVER] nsd and memory leaks
On Thu, 8 Jan 2004, Dossy wrote: > On 2004.01.08, John Shafto <[EMAIL PROTECTED]> wrote: > > I was running nsd v.3.4.2 on a fairly active website > > (FreeBSD 4.x os) for a few weeks and had some > > trouble with the nsd process growing. I was restarting > > the process every few days as it grew to 40-60Mb. > > 40-60MB is nothing. I'd worry if your nsd grows beyond 2GB. Disable the caches. You don't need them for a site that only gets a few hits a day. The configuration below runs on a P2-400 and can still serve 40 pages/second: wwwrun1056 0.0 1.3 12296 3556 ?S 2003 0:00 /opt/aolserver/bin/nsd -ft /opt/aolserver/nsd.tcl -u wwwrun wwwrun1059 0.0 1.3 12296 3556 ?S 2003 0:01 /opt/aolserver/bin/nsd -ft /opt/aolserver/nsd.tcl -u wwwrun wwwrun1060 0.0 1.3 12296 3556 ?S 2003 0:00 /opt/aolserver/bin/nsd -ft /opt/aolserver/nsd.tcl -u wwwrun wwwrun1078 0.0 1.3 12296 3556 ?S 2003 0:01 /opt/aolserver/bin/nsd -ft /opt/aolserver/nsd.tcl -u wwwrun wwwrun 0.0 1.3 12296 3556 ?S 2003 0:00 /opt/aolserver/bin/nsd -ft /opt/aolserver/nsd.tcl -u wwwrun wwwrun1166 0.0 1.3 12296 3556 ?S 2003 0:00 /opt/aolserver/bin/nsd -ft /opt/aolserver/nsd.tcl -u wwwrun wwwrun1167 0.0 1.3 12296 3556 ?S 2003 2:46 /opt/aolserver/bin/nsd -ft /opt/aolserver/nsd.tcl -u wwwrun wwwrun 18918 0.0 1.3 12296 3556 ?S10:29 0:00 /opt/aolserver/bin/nsd -ft /opt/aolserver/nsd.tcl -u wwwrun The nsd process is running for 64 days now. 12mb is not much at all. In fact, the Apache that does run on the same machine: root 1193 0.0 0.0 97424 140 ?R 2003 0:39 /usr/sbin/httpd -f /etc/httpd/httpd.conf wwwrun 13427 0.2 1.4 98212 3600 ?SJan16 8:02 /usr/sbin/httpd -f /etc/httpd/httpd.conf wwwrun 13441 0.1 1.5 98336 3932 ?SJan16 6:20 /usr/sbin/httpd -f /etc/httpd/httpd.conf wwwrun 18952 0.0 0.5 97468 1424 ?S10:30 0:00 /usr/sbin/httpd -f /etc/httpd/httpd.conf wwwrun 18953 1.4 1.4 98084 3704 ?S10:30 0:00 /usr/sbin/httpd -f /etc/httpd/httpd.conf wwwrun 18954 0.0 0.6 97448 1676 ?S10:30 0:00 /usr/sbin/httpd -f /etc/httpd/httpd.conf wwwrun 18956 0.0 0.3 97424 804 ?S10:30 0:00 /usr/sbin/httpd -f /etc/httpd/httpd.conf ... and those are processes instead of threads; i.e. they do not share all of their memory. It's just a normal Apache configuration, even with a few modules removed. Php is installed however, and it is really memory hungry. Daniël Mantione -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] nsd and memory leaks
Okay. From the discussion I was under the impression that each thread had its own arena, so reinitializing would free the entire arena. (I'm talking about a big block of virtual memory when I say arena.) An option would be to give each thread its own private arena, with the size configurable at runtime - say 10MB for example. Any allocations that couldn't be service from the private arena would overflow to the shared arena. When the thread exits, the entire 10MB arena is freed. The AS process size might be bigger, but a significant amount of that would probably be unaccessed VM. Combined with killing idle threads, this might lead to a more stable memory configuration. The only heap that would be subject to fragmentation would be the shared heap. Maybe there are allocators that already do something like this and I just need to go read/benchmark more code. :) This fragmentation issue isn't a big deal for us because of the daily restart, but in a 24-hour period, our dynamic content server grows to 3x its initial size because of fragmentation. Jim > > In a message dated 1/11/2004 12:03:36 AM Eastern Standard Time, > [EMAIL PROTECTED] writes: > > It seems that one way to address this memory/heap problem is to force > threads to exit after a certain amount of time idle, freeing their > entire heap area. There is code in AS to do that, but I've never > gotten it to work successfully because AS tends to "round-robin" the > workload to all threads. So they never idle. > Hi, > Unfortunately this isn't possible -- all the memory used by a thread is > simply moved over to the shared area on thread exit. That memory is reused by > other threads when needed but the fragmentation remains. To reduce > fragmentation the allocator would have to coalesce unused blocks, look for large > contiguous free zone's, and munmap those region. This garbage collection would > require all memory allocation in all threads to stop while such a scan took place > which would defeat the free-threaded design of the allocator. And, the > possibility of such a garbage collection phase would require locking on all > allocations to check if and when it had started. > -Jim -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] nsd and memory leaks
In a message dated 1/11/2004 12:03:36 AM Eastern Standard Time, [EMAIL PROTECTED] writes: It seems that one way to address this memory/heap problem is to force threads to exit after a certain amount of time idle, freeing their entire heap area. There is code in AS to do that, but I've never gotten it to work successfully because AS tends to "round-robin" the workload to all threads. So they never idle. Hi, Unfortunately this isn't possible -- all the memory used by a thread is simply moved over to the shared area on thread exit. That memory is reused by other threads when needed but the fragmentation remains. To reduce fragmentation the allocator would have to coalesce unused blocks, look for large contiguous free zone's, and munmap those region. This garbage collection would require all memory allocation in all threads to stop while such a scan took place which would defeat the free-threaded design of the allocator. And, the possibility of such a garbage collection phase would require locking on all allocations to check if and when it had started. -Jim -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] nsd and memory leaks
On 2004.01.10, Jim Wilcoxson <[EMAIL PROTECTED]> wrote: > It seems that one way to address this memory/heap problem is to force > threads to exit after a certain amount of time idle, freeing their > entire heap area. There is code in AS to do that, but I've never > gotten it to work successfully because AS tends to "round-robin" the > workload to all threads. So they never idle. Interesting -- you're saying the request dispatcher puts threads into a queue and tasks things LIFO? Sounds right, but I'd have to look at the source again ... > If threads went to the top of a work queue when they were fininshed > instead of the bottom, and threads were always removed from the top, > the idle thread exit code would stand a chance of working. Also, a > system over-configured for 30 threads would automatically adjust > itself to use only 5-6 threads, or whatever the actual load required; > 5-6 fragmented heaps is better than 30. The trade-off is that LIFO scheduling, while it doesn't produce idle threads ... is that it doesn't produce idle threads. Also, selecting which thread to hand a request off to is an O(1) operation, as well as returning the thread to the pool is also O(1). Pushing threads back on top of the queue (so that threads at the end of the queue eventually will exceed an idle threshhold) may not necessarily have the same performance characteristics. Heap fragmentation IS annoying. Perhaps if there was an ns_* API that let you mark a thread for termination and cleanup, you could couple that with [ns_info threads] to periodically flush "old" threads and have them replaced with fresh new ones. Similar to the Apache config. setting of "MaxClients" - after handilng X number of requests, the child dies and the parent forks a new one to continue handling requests. -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ "He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on." (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] nsd and memory leaks
Here's a script to dump out the memory pool info:
set text ""
set elname "blocksize nfree nget nput nrequest nlock nwait"
foreach pel [ns_info pools] {
append text "\nPoolname: [lindex $pel 0]\n"
set i 0
foreach el [lindex $pel 1] {
append text " [lindex $elname $i]: $el\n"
incr i
}
}
ns_return 200 text/plain $text
return
It seems that one way to address this memory/heap problem is to force
threads to exit after a certain amount of time idle, freeing their
entire heap area. There is code in AS to do that, but I've never
gotten it to work successfully because AS tends to "round-robin" the
workload to all threads. So they never idle.
If threads went to the top of a work queue when they were fininshed
instead of the bottom, and threads were always removed from the top,
the idle thread exit code would stand a chance of working. Also, a
system over-configured for 30 threads would automatically adjust
itself to use only 5-6 threads, or whatever the actual load required;
5-6 fragmented heaps is better than 30.
Jim
> In a message dated 1/10/04 12:03:58 AM, [EMAIL PROTECTED] writes:
>
>
> > A dynamic content server tends to grow in our
> > experience, and I've always assumed that's because of heap
> > fragmentation problems - not necessarily leaks.=A0 Just restart your
> > server once a day/week.
> >=20
>
>
> Yes -- memory fragementation is a problem in AOLserver which does lead to=20
> "large" processes. Actually, it's more accurate to say we accept fragmenta=
> tion=20
> as a side effect of Tcl's threaded memory allocator which is a per-thread,=20
> power-of-2 alloctor. This allocator essentially eliminates deadly lock=20
> contention problems of most normal allocators but requires high overhead and=
> leads to=20
> fragmentation. You can peak into the current state of the allocator with t=
> he=20
> "ns_info pools" command.
>
> It's possible to compile Tcl without the threaded allocator by editing Tcl's=
> =20
> Makefile to remove the -DUSE_THREAD_ALLOC=3D1. Then you could try alternat=
> e=20
> malloc packages which may provide better behavior for you app., i.e.:
>
> normal config:
>ns_malloc -> Tcl_Alloc -> Tcl threaded allocator -- malloc/Tcl_Alloc not=20
> compatible
>
> disabled config:
>ns_malloc -> Tcl_Alloc -> malloc -- you can pre-load some other malloc=20
> shared library
>
>
> HORD is often cited as a good alternatiive: =20
> http://www.cs.umass.edu/~emery/hoard/. =20
>
> -Jim
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with
> the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field
> of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of
your email blank.
Re: [AOLSERVER] nsd and memory leaks
In a message dated 1/10/04 12:03:58 AM, [EMAIL PROTECTED] writes: A dynamic content server tends to grow in our experience, and I've always assumed that's because of heap fragmentation problems - not necessarily leaks. Just restart your server once a day/week. Yes -- memory fragementation is a problem in AOLserver which does lead to "large" processes. Actually, it's more accurate to say we accept fragmentation as a side effect of Tcl's threaded memory allocator which is a per-thread, power-of-2 alloctor. This allocator essentially eliminates deadly lock contention problems of most normal allocators but requires high overhead and leads to fragmentation. You can peak into the current state of the allocator with the "ns_info pools" command. It's possible to compile Tcl without the threaded allocator by editing Tcl's Makefile to remove the -DUSE_THREAD_ALLOC=1. Then you could try alternate malloc packages which may provide better behavior for you app., i.e.: normal config: ns_malloc -> Tcl_Alloc -> Tcl threaded allocator -- malloc/Tcl_Alloc not compatible disabled config: ns_malloc -> Tcl_Alloc -> malloc -- you can pre-load some other malloc shared library HORD is often cited as a good alternatiive: http://www.cs.umass.edu/~emery/hoard/. -Jim 2">
Re: [AOLSERVER] nsd and memory leaks
> On 2004.01.09, Andrew Piskorski <[EMAIL PROTECTED]> wrote: > > On Fri, Jan 09, 2004 at 12:31:39PM -0800, Jim Wilcoxson wrote: > > > > > fragmentation problems - not necessarily leaks. Just restart your > > > server once a day/week. > > > > Which is still quite frequently, and thus very conservative. From > > what I've heard many people don't restart their AOLservers for months. > > I tend to schedule mine to restart once a week just to be safe. > > $ ps -A -olstart,vsz,args | grep dossy.org > Sun Nov 16 00:14:54 2003 32512 /home/aolserver/bin/nsd [...] > > A month and a half without being restarted and it's only 32MB large, AND > it's using the supposedly leaky (or unpatched) version of nscgi. # ps -A -olstart,vsz,args | grep nsd Thu Jan 30 18:19:55 2003 134064 /opt/chroot_ext/nsd [...] No need to restart it since then, as we use some ns_eval construct to reload our tcl-api-files should they change (and no nscgi, but lot's of nsv-stuff and postgres). The first few weeks I was concerned about memory leaks, as after 47 days the size of nsd was about 62 MB, two days later 76 MB, but, finally it tended to that 130MB shown above (I just found a file where I did some statistics with top). # uptime 11:11am up 388 days, 15:05, 1 user, load average: 0.00, 0.00, 0.00 ok, this particular webserver is serving only about 5500 visits/month, but I'm really very happy that AS can be so "carefree"... The only moment you can never foresee is when AS grabs twice as much memory because of its high water mark principle. But until it's not running the OS into swap space it also doesn't matter. cu Bernd. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] nsd and memory leaks
On 2004.01.09, Andrew Piskorski <[EMAIL PROTECTED]> wrote: > On Fri, Jan 09, 2004 at 12:31:39PM -0800, Jim Wilcoxson wrote: > > > fragmentation problems - not necessarily leaks. Just restart your > > server once a day/week. > > Which is still quite frequently, and thus very conservative. From > what I've heard many people don't restart their AOLservers for months. > I tend to schedule mine to restart once a week just to be safe. $ ps -A -olstart,vsz,args | grep dossy.org Sun Nov 16 00:14:54 2003 32512 /home/aolserver/bin/nsd [...] A month and a half without being restarted and it's only 32MB large, AND it's using the supposedly leaky (or unpatched) version of nscgi. Granted, it's not terribly high traffic, but it's certainly stable. -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ "He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on." (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] nsd and memory leaks
On Fri, Jan 09, 2004 at 12:31:39PM -0800, Jim Wilcoxson wrote: > fragmentation problems - not necessarily leaks. Just restart your > server once a day/week. Which is still quite frequently, and thus very conservative. From what I've heard many people don't restart their AOLservers for months. I tend to schedule mine to restart once a week just to be safe. -- Andrew Piskorski <[EMAIL PROTECTED]> http://www.piskorski.com/ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] nsd and memory leaks
My personal opinion is that one of the responses to this simple question was pretty shitty, and totally inaccurate. In our experience, an image-serving copy of AOLserver with 30 threads configured runs in about 25MB, regardless of the load, and serving dynamic content on 30 threads takes 100-200MB, depending on the amount of caching you're doing with ns_shares and other types of persistent in-memory data storage, and the size of your preloaded tcl modules library. Of course, if you configure your fastpath cache to be huge or make some other drastic config change in nsd.tcl, it will show up as increased process size. 30 threads is a lot; most sites won't need that, but it all depends on the size of your OS's socket buffers, average size of the content you're serving, etc. An image-serving AOLserver, with TCL disabled, tends to be very stable memory-wise. A dynamic content server tends to grow in our experience, and I've always assumed that's because of heap fragmentation problems - not necessarily leaks. Just restart your server once a day/week. Jim > > --- John Shafto <[EMAIL PROTECTED]> wrote: > > > > Doing what? > > This particular machine only has a 512mb of ram, > > and serves mostly static content. I have bigger > > plans for it though. > > > > > AOLserver is NOT Apache. Get used to that. > > > > I'll try to keep them straight. Thanks for the > > tip. > > Am I the only one who feels somewhat uncomfortable by > this response? > If I've only got 50mb of static content that it's > serving up, and then pushing some db stuff through the > back, what on earth would possibly make the process > use 2gb+? > > I'd be getting worried then as well... > > > = > -- > live- http://www.thedenofsin.org/ > to- AIM: IMFDUP > _-jupiter accepts your offer-_ > > > -- > AOLserver - http://www.aolserver.com/ > > To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with > the > body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field > of your email blank. > -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] nsd and memory leaks
On Fri, Jan 09, 2004 at 11:22:50AM -0800, Nathan Seven wrote: > If I've only got 50mb of static content that it's serving up, and > then pushing some db stuff through the back, what on earth would > possibly make the process use 2gb+? Nothing. I think Dossy was being facetious. Most people using AOLserver are using it for generating dynamic content, and in that scenario AOLserver using 30 to 150 mb or so of RAM is utterly normal and nothing to worry about. You can do things to reduce the memory footprint if you want, but unless it's over a few hundred MB most people don't care, as massive amounts of RAM are so cheap these days. The truly heinous memory leaks in some versions of Oracle's 9i client libraries are the only things I've heard of that, in practice, have ever made AOLserver suck up 1.5+ GB of memory. One user recently reported his AOLserver going up to 1.6 GB for exactly that reason (bad Oracle libraries) before finally crashing due to resource exhaustion on his Linux box. I've never heard of anyone seeing 2 GB or more, although no doubt there are unusual purposes and configurations that could legitimately use that much RAM. -- Andrew Piskorski <[EMAIL PROTECTED]> http://www.piskorski.com/ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] nsd and memory leaks
--- John Shafto <[EMAIL PROTECTED]> wrote: > > Doing what? > This particular machine only has a 512mb of ram, > and serves mostly static content. I have bigger > plans for it though. > > > AOLserver is NOT Apache. Get used to that. > > I'll try to keep them straight. Thanks for the > tip. Am I the only one who feels somewhat uncomfortable by this response? If I've only got 50mb of static content that it's serving up, and then pushing some db stuff through the back, what on earth would possibly make the process use 2gb+? I'd be getting worried then as well... = -- live- http://www.thedenofsin.org/ to- AIM: IMFDUP _-jupiter accepts your offer-_ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] nsd and memory leaks
> > >[Dossy] > > > 40-60MB is nothing. I'd worry if your nsd grows beyond 2GB. > > > > Doing what? > > Just about anything. If your stacksize is set to, say, 1 MB ... and > you've got 20 threads for handling connections, you're looking at a nsd > footprint of at least 20 MB. 40-60 MB is very reasonable for a site of > any non-trivial amount of traffic. Okay, that sounds reasonable, but 2GB? What kind of work load would drive an nsd process up into the gigabytes of memory usage? > > This particular machine only has a 512mb of ram, and serves mostly > > static content. I have bigger plans for it though. > > Look at the nsd.tcl config. ... see what different settings are set to. > It's very believable that an out-of-the-box config would produce an nsd > in the 40-60 MB ballpark. That would be tolerable on a fairly loaded site, doing goodly amounts of database stuff and such. The site I am testing my first production AOLserver on is only doing mild static stuff with a few light cgi calls. I wondered where (if) it was going to stop growing. The way it was creeping looked suspiciously like a leak to me, which appears to have been confirmed by Scott. (I rebuilt with 'Ns_DStringInit(&dsPtr);' banged out of nscgi.c, and have been running it for four hours. It's sitting, apparently stable, at 14.5mb now.) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] nsd and memory leaks
On Thu, 2004-01-08 at 21:33, John Shafto wrote: > > > I was running nsd v.3.4.2 on a fairly active website > > > (FreeBSD 4.x os) for a few weeks and had some > > > trouble with the nsd process growing. I was restarting > > > the process every few days as it grew to 40-60Mb. > > > > 40-60MB is nothing. I'd worry if your nsd grows beyond 2GB. > > Doing what? > This particular machine only has a 512mb of ram, one of my sites is running AOLServer 4.0 CVS (GM keeps hanging in schedule_timeout for me), OpenACS 5.0 beta 4, and sits at roughly 125mb in use after 3 weeks. 40k uniques/day Another site running AOLServer 4.0 CVS/OpenACS 5.0 beta 1 sits at 98mb in use after 23 days, an AOLServer 3/OpenACS 4.6.3 sits at 72mb after 47 days. I don't think 60mb is out of line. By the same token, I do have apache machines burning 200mb serving less traffic than the 125mb nsd that I have. So, it all depends on what you're doing. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] nsd and memory leaks
On 2004.01.08, John Shafto <[EMAIL PROTECTED]> wrote: > > 40-60MB is nothing. I'd worry if your nsd grows beyond 2GB. > > Doing what? Just about anything. If your stacksize is set to, say, 1 MB ... and you've got 20 threads for handling connections, you're looking at a nsd footprint of at least 20 MB. 40-60 MB is very reasonable for a site of any non-trivial amount of traffic. > This particular machine only has a 512mb of ram, and serves mostly > static content. I have bigger plans for it though. Look at the nsd.tcl config. ... see what different settings are set to. It's very believable that an out-of-the-box config would produce an nsd in the 40-60 MB ballpark. -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ "He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on." (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] nsd and memory leaks
> > I was running nsd v.3.4.2 on a fairly active website > > (FreeBSD 4.x os) for a few weeks and had some > > trouble with the nsd process growing. I was restarting > > the process every few days as it grew to 40-60Mb. > > 40-60MB is nothing. I'd worry if your nsd grows beyond 2GB. Doing what? This particular machine only has a 512mb of ram, and serves mostly static content. I have bigger plans for it though. > > Maybe I'm just being paranoid and not letting it grow > > to where it needs to be (if it caches). I'm used to > > apache processes running at <20Mb. > > AOLserver is NOT Apache. Get used to that. I'll try to keep them straight. Thanks for the tip. -- Dulcius Ex Asperis -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] nsd and memory leaks
On 2004.01.08, John Shafto <[EMAIL PROTECTED]> wrote: > I was running nsd v.3.4.2 on a fairly active website > (FreeBSD 4.x os) for a few weeks and had some > trouble with the nsd process growing. I was restarting > the process every few days as it grew to 40-60Mb. 40-60MB is nothing. I'd worry if your nsd grows beyond 2GB. > Maybe I'm just being paranoid and not letting it grow > to where it needs to be (if it caches). I'm used to > apache processes running at <20Mb. AOLserver is NOT Apache. Get used to that. -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ "He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on." (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] nsd and memory leaks
Almost -- you need the -r flag to identify the branch: cvs -d:pserver:[EMAIL PROTECTED]:/cvsroot/aolserver login cvs -z3 -d:pserver:[EMAIL PROTECTED]:/cvsroot/aolserver \ co -r aolserver_v35_bp aolserver /s. On Jan 8, 2004, at 6:16 PM, John Shafto wrote: Thanks Scott, Yes, I am running some small cgi stuff, so that is likely it. I didn't see a tarball for v3.5.9 on the sourceforge site, and not being much of a manual CVSer, not sure how I would checkout v35bp. Would it be: cvs -d:pserver:[EMAIL PROTECTED]:/cvsroot/aolserver co aolserver_v35_bp ? -- Untied we stand, fettered we fall. - Original Message - From: "Scott Goodwin" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Thursday, January 08, 2004 15:37 Subject: Re: [AOLSERVER] nsd and memory leaks No, you're not being paranoid. Are you using nscgi and running CGI scripts? If so, then you're running into this problem (from the ChangeLog): 2003-04-04 Scott S. Goodwin <[EMAIL PROTECTED]> * nscgi/nscgi.c: (bug) Ns_DStringInit called in CgiExec was destroying the linked list used to maintain the list of dstrings in use by the module, resulting in a memory leak for every cgi script run. You can get AOLserver 3.5.9 or later (better yet, go grab the latest CVS copy on the v35bp branch), or go into nscgi.c and remove the line that reads: Ns_DStringInit(&dsPtr); from the CgiExec function, and recompile. /s. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] nsd and memory leaks
On 09/01/2004, at 10:16 AM, John Shafto wrote: I didn't see a tarball for v3.5.9 on the sourceforge site, and not being much of a manual CVSer, not sure how I would checkout v35bp. Would it be: cvs -d:pserver:[EMAIL PROTECTED]:/cvsroot/aolserver co aolserver_v35_bp John, I think it's: cvs -d:pserver:[EMAIL PROTECTED]:/cvsroot/aolserver co -r aolserver_v35_bp aolserver Matthew -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] nsd and memory leaks
Thanks Scott, Yes, I am running some small cgi stuff, so that is likely it. I didn't see a tarball for v3.5.9 on the sourceforge site, and not being much of a manual CVSer, not sure how I would checkout v35bp. Would it be: cvs -d:pserver:[EMAIL PROTECTED]:/cvsroot/aolserver co aolserver_v35_bp ? -- Untied we stand, fettered we fall. - Original Message - From: "Scott Goodwin" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Thursday, January 08, 2004 15:37 Subject: Re: [AOLSERVER] nsd and memory leaks No, you're not being paranoid. Are you using nscgi and running CGI scripts? If so, then you're running into this problem (from the ChangeLog): 2003-04-04 Scott S. Goodwin <[EMAIL PROTECTED]> * nscgi/nscgi.c: (bug) Ns_DStringInit called in CgiExec was destroying the linked list used to maintain the list of dstrings in use by the module, resulting in a memory leak for every cgi script run. You can get AOLserver 3.5.9 or later (better yet, go grab the latest CVS copy on the v35bp branch), or go into nscgi.c and remove the line that reads: Ns_DStringInit(&dsPtr); from the CgiExec function, and recompile. /s. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] nsd and memory leaks
No, you're not being paranoid. Are you using nscgi and running CGI scripts? If so, then you're running into this problem (from the ChangeLog): 2003-04-04 Scott S. Goodwin <[EMAIL PROTECTED]> * nscgi/nscgi.c: (bug) Ns_DStringInit called in CgiExec was destroying the linked list used to maintain the list of dstrings in use by the module, resulting in a memory leak for every cgi script run. You can get AOLserver 3.5.9 or later (better yet, go grab the latest CVS copy on the v35bp branch), or go into nscgi.c and remove the line that reads: Ns_DStringInit(&dsPtr); from the CgiExec function, and recompile. /s. On Jan 8, 2004, at 4:57 PM, John Shafto wrote: I was running nsd v.3.4.2 on a fairly active website (FreeBSD 4.x os) for a few weeks and had some trouble with the nsd process growing. I was restarting the process every few days as it grew to 40-60Mb. I posted a couple messages on this list about v4.0 and using port 80 as I was trying to run it, but decided it's probably not stable enough for a production server. I built v3.5.6 for the site with the latest tcl (8.4.5) and the PHP4.3.4 sapi module (but not running any php actively on the site). I'm running zero dynamic/database stuff at this point, but am running a couple small perl/C cgi things (form mailer and counter). Now, it appears that the 3.5.6 process still has a memory leak, as it has been steadily growing for about the last 24 hours. It's up to 21mb, which would be okay if it stops growing pretty soon. Anyone have any ideas on what causes or how to stop memory leaks in version 3.x? What size do your nsd processes typically run at after a few days? Maybe I'm just being paranoid and not letting it grow to where it needs to be (if it caches). I'm used to apache processes running at <20Mb. Thanks, -- Untied we stand, fettered we fall. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
