Re: [AOLSERVER] Lengthy tcl interp initialization
On 2003.08.13, Elizabeth Thomas [EMAIL PROTECTED] wrote: We have a server that loads in a great deal of tcl - so much so that the resulting init script is 6.6M in size and contains over 9900 procs. [...] 1. Is there any way to reduce the time to initialize an interp? (besides the obvious, but not necessarily feasible, option of reducing the procs that are loaded). Has anyone seen similar behavior and have some insight into it? Is there any chance to do further profiling? Where are we losing most of our time? I imagine it has a lot to do with allocating 6.6M of memory for the init script, filling it with the script, then telling Tcl to go parse and execute it. I think the biggest win would be to try and figure out how to get Tcl to bytecode compile the init script, then push that bytecode from the master interp into the new slave interps as they get created. Cut out the entire parse/bytecode compile steps. Perhaps someone who knows and understand those intricate details of Tcl 8.4 can speak up? -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on. (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] ns_include -sameframe
On 2003.08.14, Nikolay Skachkov [EMAIL PROTECTED] wrote:
I am in a process of updating some application still running under
aolserver 2.3 to 3.5. That has a lot of statements ns_include -sameframe
The -sameframe option is not supported any longer by the api and I well
understand why. The recommended way of passing parameters to includes is
via explicit list and then ns_adp_bind_args.
However moving to new api with my application will mean a huge rewrite.
Is there any way to avoid that?
Rename the current ns_include proc to _std_ns_include or somesuch, then
define your own ns_include proc that looks for the -sameframe arg and
does something intelligent with it ...
However, all the code that gets included will need to be touched to be
wrapped in an uplevel (simplest change, can be applied
programmatically).
Basically, write a script that greps all your .adp files for -sameframe.
Build up a unique list of filenames that get included using -sameframe.
Then, write a script that takes each of those files, reads in their
contents, and writes them back out with the file contents in uplevel
{original file contents} ...
Then, strip out the -sameframe using perl -pi -e s/ -sameframe//g'
*.adp or whatnot.
Of course, take a full backup of all your files first in case you need
to back out of this change if it doesn't work.
Huge rewrite -- how many files/lines of code are we talking here?
-- Dossy
--
Dossy Shiobara mail: [EMAIL PROTECTED]
Panoptic Computer Network web: http://www.panoptic.com/
He realized the fastest way to change is to laugh at your own
folly -- then you can let go and quickly move on. (p. 70)
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of
your email blank.
Re: [AOLSERVER] Lengthy tcl interp initialization
We are currently pursuing a very encouraging approach of adding an optional 'lazy proc definition' capability, capitalizing on the 'unknown' processing of tcl. (Thanks to Jeff Hobbes for putting us on this path). Since most of our threads use a relatively small subset of all available procs, we hope to achieve significant performance and memory consumption wins by only loading procs in the interpeter that are actually needed. More details to come early next week. -Elizabeth Dossy wrote: On 2003.08.13, Elizabeth Thomas [EMAIL PROTECTED] wrote: We have a server that loads in a great deal of tcl - so much so that the resulting init script is 6.6M in size and contains over 9900 procs. [...] 1. Is there any way to reduce the time to initialize an interp? (besides the obvious, but not necessarily feasible, option of reducing the procs that are loaded). Has anyone seen similar behavior and have some insight into it? Is there any chance to do further profiling? Where are we losing most of our time? I imagine it has a lot to do with allocating 6.6M of memory for the init script, filling it with the script, then telling Tcl to go parse and execute it. I think the biggest win would be to try and figure out how to get Tcl to bytecode compile the init script, then push that bytecode from the master interp into the new slave interps as they get created. Cut out the entire parse/bytecode compile steps. Perhaps someone who knows and understand those intricate details of Tcl 8.4 can speak up? -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on. (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Lengthy tcl interp initialization
On Fri, 15 Aug 2003, Elizabeth Thomas wrote: We are currently pursuing a very encouraging approach of adding an optional 'lazy proc definition' capability, capitalizing on the 'unknown' processing of tcl. (Thanks to Jeff Hobbes for putting us on this path). Since most of our threads use a relatively small subset of all available procs, we hope to achieve significant performance and memory consumption wins by only loading procs in the interpeter that are actually needed. An approach like this was discussed on this list a couple of years ago, and Rob Mayoff had particular suggestions for handling namespaces. At the time, if I recall, some AOLserver mods were necessary to support proper handlikng of namespaces, but those are probably obsolete now. Also, at the time, Jim felt the approach was not necessary for AOL, so it didn't make it into the default thread initialization. The old notes are probably in the list archives. Pete. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Lengthy tcl interp initialization
On 2003.08.15, Elizabeth Thomas [EMAIL PROTECTED] wrote:
We are currently pursuing a very encouraging approach of adding an
optional 'lazy proc definition' capability, capitalizing on the
'unknown' processing of tcl. (Thanks to Jeff Hobbes for putting us on
this path). Since most of our threads use a relatively small subset of
all available procs, we hope to achieve significant performance and
memory consumption wins by only loading procs in the interpeter that are
actually needed.
More details to come early next week.
This works for procs, but what about the rest of the interp init code?
Suppose you have this in your interp init script:
foreach procName [list a b c d] {
proc foo_$procName args
eval some_proc $procName \$args
}
Pretty simple, you'd only worry about foo_a through foo_d, regardless of
how they came to be defined.
But, what about:
global x
set x 0
foreach x [list 1 2 3 4 5] {
proc foo_$x args
global x
format {the last proc defined is %u} \$x
}
I really wish I had /real/ code to quote here, but the point is: there
CAN be code that doesn't live within a proc definition in your interp
scripts that have side-effects that matter.
-- Dossy
--
Dossy Shiobara mail: [EMAIL PROTECTED]
Panoptic Computer Network web: http://www.panoptic.com/
He realized the fastest way to change is to laugh at your own
folly -- then you can let go and quickly move on. (p. 70)
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of
your email blank.
Re: [AOLSERVER] Lengthy tcl interp initialization
On Friday 15 August 2003 17:44, you wrote: This works for procs, but what about the rest of the interp init code? The blueprint script can be modified to store plain proc definitions (Part1) apart from any other non-proc code (Part2) Then load the Part2 always and use unknown to fish out proc definitions from Part1. The defs can be retrieved from the master interp easily during runtime on the fly. Would require some internal changes to the Tcl init code and one or two new ns_ commands but it may work. On the first glance, I like the idea. One may also make use of nsv to avoid changing internals. Zoran -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Lengthy tcl interp initialization
Just as an alternative approach have you thought of breaking your 9900 procs into separate nsds that tackle separate problems and have a proxy server out front of your nsd that route the call to the appropriate nsd. For example if your app has a message board part . only load the common procs and the message board procs for that server. Then configure the proxy/load balancer to direct all call to the message boards to that nsd. I'm sure you are already multiple machines, right? Many load balancers now can direct the request by Url in addition to Round Robin, Weighted, Least Connection, etc. Just an idea, Carl Garland _ Add photos to your messages with MSN 8. Get 2 months FREE*. http://join.msn.com/?page=features/featuredemail -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] ns_include -sameframe
Rename the current ns_include proc to _std_ns_include or somesuch, then define your own ns_include proc that looks for the -sameframe arg and does something intelligent with it ... However, all the code that gets included will need to be touched to be wrapped in an uplevel (simplest change, can be applied programmatically). I guess I am not following here (maybe because its friday), but why do you need the uplevel? Couldn't the new proc do this. I have never used ns_include... --brett __ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
