Re: [AOLSERVER] 3.4, idle thread timeout broken on Linux
This shows (I think) that if you currently have N threads active, no threads will timeout if there are N hits in threadtimeout seconds, at least on a Linux box, because the thread scheduling is FIFO instead of LIFO. On one of our production server, we are up to around 26 active threads. I don't think there is any value for threadtimeout that would make threads actually time-out. That could be, I don't know how Linux wakes things up, BUT, I dunno if there is a way to make Linux/pthreads/AS/... wakeup threads in LIFO order instead of FIFO. Anyone else know? From what I recall of how threads are started and killed, well, if this is a concern of yours, then another way to skin that cat might to alter the logic in nsthread and conn.c maintaining a ratio of active threads to total threads, and to start killing threads off when the ratio becomes too low. Jerry Jerry Asher [EMAIL PROTECTED] 1678 Shattuck Avenue Suite 161 Tel: (510) 549-2980 Berkeley, CA 94709 Fax: (877) 311-8688
Re: [AOLSERVER] MySQL/AOLServer - Strange things...
Hi !
I was afraid that you would say to recompile ;-) I need to revise my C++!
Thanks for your help, and I will try this evening!
- Daniel
- Original Message -
From: Ariel E. Carná [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Thursday, October 18, 2001 3:48 AM
Subject: Re: [AOLSERVER] MySQL/AOLServer - Strange things...
Hi Daniel!
Some time ago I have the same problem trying to load the Oracle driver
with
AOLServer 3.4 for Windows.
I have to touch the source files and compile all again.
The file that I have to modify was ns.h, search this section of code and
try to modify and compile again.
The type long long is supported by VC6 but something is different with
__int64, I don't know exactly where
is the difference (both types are 64 bits and integers) but with this
modification, AOLServer and the driver
for Oracle work fine.
Good luck!
Ariel.
#define NS_SOCK_READ1
#define NS_SOCK_WRITE2
#define NS_SOCK_EXCEPTION 4
#define NS_SOCK_EXIT8
#define NS_NO_DATA 8
#define NS_END_DATA 4
#define NS_ROWS 2
#define NS_DML 1
#define NS_ENCRYPT_BUFSIZE 16
/* Ariel */
#ifndef WIN32
#ifdef __alpha
typedef long INT64;
#define NS_INT_64_FORMAT_STRING %ld
#else
typedef long long INT64;
#define NS_INT_64_FORMAT_STRING %lld
#endif
#else
/* Ariel */
/* typedef long INT64; */
typedef __int64 INT64;
#endif
#define NS_TCL_SET_TEMPORARY 0
#define NS_TCL_SET_DYNAMIC1
#define NS_TCL_SET_PERSISTENT 2
- Original Message -
From: Daniel Page [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Wednesday, October 17, 2001 8:25 AM
Subject: [AOLSERVER] MySQL/AOLServer - Strange things...
Hi,
My final tests were running on a WinNT 4 box. I copied the nsmysql.so
(renamed to nsmysql.dll) file to the same directory as nscp.dll
(c:/progra~1/aolserver/bin), and in the config file, and I copied the
entry
for ${bindir}/nscp${ext} in ns/server/${servername}/modules to
${bindir}/nsmysql${ext} in ns/db/drivers - The nscp.dll module loads, but
nsmysql.dll won't, still with the error 126 (file not found)...
I then inversed the names of nscp.dll and nsmysql.dll files on the
disk -
From this point, nsmysql.dll file loads, but gives an error about
ns_dbDriverInit - this seems normal, as it is a renamed console module,
but
the important point is that AOLServer *can* find the file named
nsmysql.dll - and that the system gives an error 126 on loading the
nscp.dll
file, which is in reality the nsmysql driver.
This proves that AOLServer *can* find the nsmysql.dlI file, and from
this,
I can deduct from this that there is a problem with the actual nsmysql
module, and that the Win32 error 126 in this case does not seem to mean
file not found.
The problem follows the *content* of the file, but not the *name* -
unless
it cannot find a file that the module itself is dependant on... in which
case, the error 126 is not clear enough... and I do not have the Microsoft
Dependancy Walker tool to hand to check at this time...
Aggghhh!!!
- Daniel
-
Nokia Game is on again.
Click here to join the new all media adventure before November 3rd.
Re: [AOLSERVER] 3.4 memory usage
Harry Moreau wrote: Personally, I'm heartened to hear other people see leaks Yea, me too! I have several systems where nsd 3.2+ad12 will slowly consume memory until, if not restarted, it will crash the system. I had been debating whether I should upgrade to 3.4 and see if it would help, but it sounds to me like the problem is still present there. janine
Re: [AOLSERVER] 3.4 memory usage
I was travelling yesterday, plus we are still fighting a few fires since the 3.4 upgrade. To answer some of the questions/suggestions people have posted: 1. Yes, I'm sure we're running 7.6 TCL. I ran into a few problems with 8.X because we (intentionally) use poorly-constructed lists in a couple of places and 8.X complains while 7.6 doesn't. Second, I didn't want to deal with the character set issues right now - sounds like a mess. Third, 7.6 is faster than 8.X in the limited testing I've done, even in CPU-bound loops. I'm assuming this is because ns_shares are more efficient in 7.6, but not sure. Here's the test I did: 2. The server that is growing is a special-purpose server. Over 90% of all requests are one of two kinds, which should make tracking down the cause a lot easier. This server may do a lot of execs in some cases, which is why I asked about that. Our main server, with a large variety of pages, seems to be doing fine memory-wise. (And hasn't crashed - YAY!) The special server does around 750K hits/day. 3. We have monitoring tools that complain if a server takes longer than 10 seconds to respond to a request. It has been complaining about this same server. I remember when I ran benchmarks with some version of AS that it would periodically go to sleep for several seconds with the CPU idle. I need to go back and see if I can duplicate that behavior because I think we are seeing it in production. 4. We don't have adp even configured, but I'll check out the fastpath stuff for memory usage. We don't serve any static pages and very few graphics from AS. Here's an interesting stat: around 80% of our graphics hits are for a group of less than 20 files. (Not 80% of the graphics data transfers mind you, but 80% of the hits) 5. In one of my posts I said this server is not accumulating data in ns_shares, but didn't mean that it doesn't use them at all. All of our servers use ns_shares - alot. What I meant was that we are not doing anything that would explain why the server would start out with 81MB with the ns_shares loaded, then grow to 240MB. We do create new ns_share array entries while it's running, but not to this extent. We don't use nsv's - IMO that programming model is broken because regular TCL constructs can't be used on nsv's. Overall, I'd have to say I'm real happy with the upgrade from 2.3.3. 3.4 is more stable and the fact that we now have the source for the server we are running is a HUGE deal. Running 2.3.3 was pretty scary business-wise, especially when AOL bought Netscape and the future of AS seemed a bit in doubt. Footnote: is anyone (or most people) using zippy? I still haven't tried that. Jim Harry Moreau wrote: Personally, I'm heartened to hear other people see leaks Yea, me too! I have several systems where nsd 3.2+ad12 will slowly ... janine
[AOLSERVER] regular Tcl constructs and nsv's? (was Re: memory usage in 3.4)
On Wednesday, October 17, 2001, at 10:23 AM, Jim Wilcoxson wrote: We don't use nsv's - IMO that programming model is broken because regular TCL constructs can't be used on nsv's. I probably missed a memo or something, but can you tell me what you have in mind here? Pete.
Re: [AOLSERVER] regular Tcl constructs and nsv's? (was Re: memory
proc foo {var} {
if {$var == 10} {
return 1
} else {
return 0
}
}
ns_share shvar
set shvar 10
set plainvar 20
if {[foo $shvar] || [foo $plainvar]} {
ns_return 200 text/plain had a foo
} else {
ns_return 200 text/plain no foos here
}
I realize that when calling foo I could dereference the nsv, but it
gets even more klunky if foo has to set one of its arguments. How
does it know if it's an nsv or not? You have to pass a flag with
every variable passed by reference and have two sets of code in the
routine: one to set the argument if it's an nsv, and another to set it
if it's a real variable. That's a huge kludge IMO. I know a good
program wouldn't change the value of arguments. But in practice it
is useful and it happens. Plus, now that we have 90K lines of TCL
using ns_shares, changing them to use nsv's would be a real nosebleed.
Jim
On Wednesday, October 17, 2001, at 10:23 AM, Jim Wilcoxson wrote:
We don't use nsv's - IMO that programming model is broken because
regular TCL constructs can't be used on nsv's.
I probably missed a memo or something, but can you tell me what you have
in mind here?
Pete.
Re: [AOLSERVER] 3.4 memory usage
Try 3.3+ad13. It has a memory leak fix involving TSDs that I back-ported from the 4.0 tree. +-- On Oct 17, Janine Sisk said: Yea, me too! I have several systems where nsd 3.2+ad12 will slowly consume memory until, if not restarted, it will crash the system.
Re: [AOLSERVER] 3.4 memory usage
In glancing at the zippy code, it looks like it used a power-of-2 algorithm, so I figured it might cause less heap fragmentation. I think that might be at least some of the problem. Does the standard gnu/linux memory allocator handle fragmentation poorly/well? +-- On Oct 17, Jim Wilcoxson said: Footnote: is anyone (or most people) using zippy? I still haven't tried that. The zippy malloc will probably use more memory but should improve performance. It should have no effect on memory leaks. I believe AOL uses it in production.
Re: [AOLSERVER] 3.4 memory usage
+-- On Oct 17, Jim Wilcoxson said: In glancing at the zippy code, it looks like it used a power-of-2 algorithm, so I figured it might cause less heap fragmentation. I think that might be at least some of the problem. Does the standard gnu/linux memory allocator handle fragmentation poorly/well? I think the standard Linux allocator is dl-malloc, which as I recall has pretty good fragmentation properties. The reason zippy may use more memory is that it keeps a separate pool of memory for each thread. This reduces lock contention but means that less free memory is shared between threads.
[AOLSERVER] Missing AOLServer DLL files under the Windows distribution...
Hi, It would seem that 2 DLL files are missing from the Windows pre-compiled version of AOLServer, downloaded from aolserver.com: - nstcl.dll - nsd.dll If anyone has a copy of these files for AOLServer 3.4, could you please send me a copy? Cheers, Daniel - Nokia Game is on again. Click here to join the new all media adventure before November 3rd.
Re: [AOLSERVER] 3.4 memory usage
Janine Sisk wrote: Harry Moreau wrote: Personally, I'm heartened to hear other people see leaks Yea, me too! I have several systems where nsd 3.2+ad12 will slowly consume memory until, if not restarted, it will crash the system. I had been debating whether I should upgrade to 3.4 and see if it would help, but it sounds to me like the problem is still present there. janine for wahtever it's worth we had problems w/ leaks in 3.2 and 3.3, mainly when threads expired. 3.4 has ben behaving for us using tcl8.x... -mike -- Mike Hoegeman Email: [EMAIL PROTECTED] Phone: 805-279-7306
[AOLSERVER] AOLserver complies with HTTP minimums?
Hi there, in Screen Name Service, we're currently investigating a problem regarding the truncation/dropping of HTTP headers when they're becoming too big or too many for the client or the server to handle. Just wanted to ask you guys if you know of any limitations of AOLserver in this regard. Especially, do you guys support the following minimums for both incoming and outgoing headers: * 4K characters per HTTP header line * 16K characters total for all HTTP header lines * 100 HTTP header lines Also, if you know of any other limitations, please let us know. Thanks for your help! -- Mike Mike Schilli [EMAIL PROTECTED] Magic Carpet Engineering
Re: [AOLSERVER] embed tag and ns_returnfile...
i'm not sure this will help but you might try making the url that generates the pdf file end with .pdf (if you not doing that already..) i found that some versions of IE don't always pay attention to the content-type but they will pay attention to the file suffix.. OK, I have pdflib going in aolserver and it generates beautiful pdf reports. I send them to the browser via 'ns_returnfile 200 application/pdf /tmp/pdffilename'. On some machines it works great. On others, it does not work at all. The root of the problem is something to do with the embed tag generated by the browser to display the file. It seems that the embed tag has a URL embedded (no pun intended) in it. I assume the browser sees the header contains a file type that it can embed, so it generates its own embed tag and sends it to itself. It then goes back to that url to get the file, taking advantage of the delay the host application (Acrobat) takes to launch to hide this silliness. If this is true, I can see where I cause it problems, I register a proc to generate a form in which the users specifies criteria for GET requests, and a proc to generate a pdf for POST requests. If the browser is issuing a GET for the url in the embed tag, that explains the problem, but not why it works on some machines but not on others. All browsers are IE5, all Acrobat is 4.0, all clients are Win 98. Of course, there are myriad patchlevels Has anyone else had issues like this with generated pdf where the GET and POST registered procs are similar to mine? I am thinking the only fix is to return a redirect to a directory where the report is instead of the pdf file itself... Thanks in advance... Ian A. Harding Programmer/Analyst II Tacoma-Pierce County Health Department (253) 798-3549 mailto: [EMAIL PROTECTED] -- Mike Hoegeman Email: [EMAIL PROTECTED] Phone: 805-279-7306
Re: [AOLSERVER] embed tag and ns_returnfile...
There are 2 MS support articles you need to read. Mike's correct in stating IE ignores MIME type - it does a lot of second-guessing. One of the articles deals with that: http://msdn.microsoft.com/library/default.asp?url=/workshop/networking/monik er/overview/appendix_a.asp Another is in the Knowledge Base Q254337 (also Q293792) explaining how astonishingly brain-dead IE can be as it sends 2 or 3 requests (contype requests) for every actual request. This can play havoc with server side programs. IMO those should at least be HEAD requests. I believe that constitutes a breach of standards (no news there of course). Hope this helps at least a little. I assume others on this list already knew about this stuff? -- Mark Hubbard: [EMAIL PROTECTED] Microsoft Certified Professional Never apply a Star Trek solution to a Babylon 5 problem. -Original Message- From: Mike Hoegeman [EMAIL PROTECTED] To: [EMAIL PROTECTED] [EMAIL PROTECTED] Date: Wednesday, October 17, 2001 12:58 PM Subject: Re: embed tag and ns_returnfile... i'm not sure this will help but you might try making the url that generates the pdf file end with .pdf (if you not doing that already..) i found that some versions of IE don't always pay attention to the content-type but they will pay attention to the file suffix.. OK, I have pdflib going in aolserver and it generates beautiful pdf reports. I send them to the browser via 'ns_returnfile 200 application/pdf /tmp/pdffilename'. On some machines it works great. On others, it does not work at all. The root of the problem is something to do with the embed tag generated by the browser to display the file. It seems that the embed tag has a URL embedded (no pun intended) in it. I assume the browser sees the header contains a file type that it can embed, so it generates its own embed tag and sends it to itself. It then goes back to that url to get the file, taking advantage of the delay the host application (Acrobat) takes to launch to hide this silliness. If this is true, I can see where I cause it problems, I register a proc to generate a form in which the users specifies criteria for GET requests, and a proc to generate a pdf for POST requests. If the browser is issuing a GET for the url in the embed tag, that explains the problem, but not why it works on some machines but not on others. All browsers are IE5, all Acrobat is 4.0, all clients are Win 98. Of course, there are myriad patchlevels Has anyone else had issues like this with generated pdf where the GET and POST registered procs are similar to mine? I am thinking the only fix is to return a redirect to a directory where the report is instead of the pdf file itself... Thanks in advance... Ian A. Harding Programmer/Analyst II Tacoma-Pierce County Health Department (253) 798-3549 mailto: [EMAIL PROTECTED] -- Mike Hoegeman Email: [EMAIL PROTECTED] Phone: 805-279-7306
Re: [AOLSERVER] 3.4 memory usage
Ok, I must have missed something, or might have been off of the cluetrain too long, but what exactly is 'zippy'? I did a google search, but I was getting mostly 'zippy the pinhead' and other weird stuff! Anyone have an URL or explanation? thanks, --brett On Wed, 17 Oct 2001 09:54:25 -0500 Rob Mayoff [EMAIL PROTECTED] wrote: +-- On Oct 17, Jim Wilcoxson said: In glancing at the zippy code, it looks like it used a power-of-2 algorithm, so I figured it might cause less heap fragmentation. I think that might be at least some of the problem. Does the standard gnu/linux memory allocator handle fragmentation poorly/well? I think the standard Linux allocator is dl-malloc, which as I recall has pretty good fragmentation properties. The reason zippy may use more memory is that it keeps a separate pool of memory for each thread. This reduces lock contention but means that less free memory is shared between threads. _ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com
Re: [AOLSERVER] 3.4 memory usage
+-- On Oct 17, Brett Schwarz said: Ok, I must have missed something, or might have been off of the cluetrain too long, but what exactly is 'zippy'? I did a google search, but I was getting mostly 'zippy the pinhead' and other weird stuff! It's the -z flag to nsd.
Re: [AOLSERVER] 3.4 memory usage
zippy is the -z command line option to AS. It causes an AOL-designed memory allocator to be used instead of the standard C library malloc. Properties of zippy are that it has separate heaps for each thread instead of a shared heap, thus avoiding the need to lock when malloc'ing private thread storage, and it uses a different alloc/freelist strategy. The wrappers to choose one vs the other are in thread/memory.c. The zippy allocator is in thread/pool.c Jim Ok, I must have missed something, or might have been off of the cluetrain too long, but what exactly is 'zippy'? I did a google search, but I was getting mostly 'zippy the pinhead' and other weird stuff! Anyone have an URL or explanation? thanks, --brett On Wed, 17 Oct 2001 09:54:25 -0500 Rob Mayoff [EMAIL PROTECTED] wrote: +-- On Oct 17, Jim Wilcoxson said: In glancing at the zippy code, it looks like it used a power-of-2 algorithm, so I figured it might cause less heap fragmentation. I think that might be at least some of the problem. Does the standard gnu/linux memory allocator handle fragmentation poorly/well? I think the standard Linux allocator is dl-malloc, which as I recall has pretty good fragmentation properties. The reason zippy may use more memory is that it keeps a separate pool of memory for each thread. This reduces lock contention but means that less free memory is shared between threads. _ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com
Re: [AOLSERVER] AOLserver complies with HTTP minimums?
This will probably be an issue for us in the future. Thanks for bringing it to light. -- Mark Hubbard: [EMAIL PROTECTED] Microsoft Certified Professional Never apply a Star Trek solution to a Babylon 5 problem. -Original Message- From: Dossy [EMAIL PROTECTED] To: [EMAIL PROTECTED] [EMAIL PROTECTED] Date: Wednesday, October 17, 2001 2:50 PM Subject: Re: AOLserver complies with HTTP minimums? On 2001.10.17, Kriston Rehberg [EMAIL PROTECTED] wrote: There is a maxpost of 65535 bytes which means 65,535 bytes total can be presented to the server in the body of a POST POST or in the query part of a GET string. I know you didn't really ask for that but it is helpful to know. Hmm, the default 65k for POST kinda sucks for people who do content management through AOLserver who allow file uploads ... Maybe making maxpost default to 10 MB or something may be useful. Then again, this doesn't seem to be a FAQ so maybe this is a non-issue for the average setup. -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on. (p. 70)
Re: [AOLSERVER] AOLserver complies with HTTP minimums?
+-- On Oct 17, Mark Hubbard said: This will probably be an issue for us in the future. Thanks for bringing it to light. Note that if you use ns_getform, and the request was a POST with content in multipart/form-data format, then maxpost does not apply.
Re: [AOLSERVER] AOLserver complies with HTTP minimums?
+-- On Oct 17, [EMAIL PROTECTED] said: Where is the limit, exactly? Tcl API or C API? The limit is in the Ns_ConnGetQuery function. It does not apply to file uploads, because those are sent in multipart/form-data format.
Re: [AOLSERVER] AOLserver complies with HTTP minimums?
+-- On Oct 17, Rob Mayoff said: The limit is in the Ns_ConnGetQuery function. It does not apply to file uploads, because those are sent in multipart/form-data format. More accurately, the limit is enforced by the Ns_ConnGetQuery function. It is set using the ns/server/servername maxpost parameter.
[AOLSERVER] 3.4, idle thread timeout broken on Linux
On a test server configured with threadtimeout set to 120, minthreads not set (defaults to 1/0 I think), and maxthreads set to 40, I have another server reference a URL every 5 seconds. What I see on the test server is: [17/Oct/2001:19:17:49][9533.8200][-conn0-] Notice: monitor: returning page [17/Oct/2001:19:17:54][9533.8200][-conn0-] Notice: monitor: returning page [17/Oct/2001:19:17:59][9533.8200][-conn0-] Notice: monitor: returning page [17/Oct/2001:19:18:04][9533.8200][-conn0-] Notice: monitor: returning page Then I go click around on the site to cause another thread to launch to handle the demand. Then stop clicking to let things settle down. Now I see this: [17/Oct/2001:19:19:55][9533.8200][-conn0-] Notice: monitor: returning page [17/Oct/2001:19:20:00][9533.14345][-conn1-] Notice: monitor: returning page [17/Oct/2001:19:20:05][9533.8200][-conn0-] Notice: monitor: returning page [17/Oct/2001:19:20:10][9533.14345][-conn1-] Notice: monitor: returning page [17/Oct/2001:19:20:15][9533.8200][-conn0-] Notice: monitor: returning page [17/Oct/2001:19:20:20][9533.14345][-conn1-] Notice: monitor: returning page [17/Oct/2001:19:20:25][9533.8200][-conn0-] Notice: monitor: returning page [17/Oct/2001:19:20:30][9533.14345][-conn1-] Notice: monitor: returning page This shows (I think) that if you currently have N threads active, no threads will timeout if there are N hits in threadtimeout seconds, at least on a Linux box, because the thread scheduling is FIFO instead of LIFO. On one of our production server, we are up to around 26 active threads. I don't think there is any value for threadtimeout that would make threads actually time-out. I dunno if there is a way to make Linux/pthreads/AS/... wakeup threads in LIFO order instead of FIFO. Anyone else know? Jim
[AOLSERVER] embed tag and ns_returnfile...
OK, I have pdflib going in aolserver and it generates beautiful pdf reports. I send them to the browser via 'ns_returnfile 200 application/pdf /tmp/pdffilename'. On some machines it works great. On others, it does not work at all. The root of the problem is something to do with the embed tag generated by the browser to display the file. It seems that the embed tag has a URL embedded (no pun intended) in it. I assume the browser sees the header contains a file type that it can embed, so it generates its own embed tag and sends it to itself. It then goes back to that url to get the file, taking advantage of the delay the host application (Acrobat) takes to launch to hide this silliness. If this is true, I can see where I cause it problems, I register a proc to generate a form in which the users specifies criteria for GET requests, and a proc to generate a pdf for POST requests. If the browser is issuing a GET for the url in the embed tag, that explains the problem, but not why it works on some machines but not on others. All browsers are IE5, all Acrobat is 4.0, all clients are Win 98. Of course, there are myriad patchlevels Has anyone else had issues like this with generated pdf where the GET and POST registered procs are similar to mine? I am thinking the only fix is to return a redirect to a directory where the report is instead of the pdf file itself... Thanks in advance... Ian A. Harding Programmer/Analyst II Tacoma-Pierce County Health Department (253) 798-3549 mailto: [EMAIL PROTECTED]
