Re: [AOLSERVER] AOLserver and Tcl Ports to iPhone

[Nathan Folkman]

Yeah, it was an interesting idea... ;)

http://www.freshpatents.com/Client-server-web-application-architectures-for-offline-usage-data-structures-and-related-methods-dt20070208ptan20070033569.php

- n

On 10/31/07, patrick o'leary <[EMAIL PROTECTED]> wrote:
>
>  It's funny, when I used work for Eric,  every time our operations /
> sysadmin group complained about disk usage & cost of hardware,
> we'd say we'd work on porting aolserver to the iPod, and slowly start
> replacing boxes with cheap iPods
>
> Emmm it was a joke Eric, just a joke. ;-)
>
> But I've actually seen a few apps developed by running aolserver on the
> desktop, instead of having to learn the rc components
> of each operating system, your display mechanism can simply be html.
>
> Google's original desktop search app was a light webserver, so aolsever
> folks weren't the only ones to think of it.
>
> p
>
> Tomasz Kosiak wrote:
>
> FYI: Eric O'Laughlen ported AOLserver and Tcl to iPhone
>
> http://velocipeek.com/2007/09/17/iphone-aolserver-and-tcl-ports/
>
>
> --tkosiak
>
> Here is a quote from his blog
>
> =
> http://velocipeek.com/2007/09/21/joggame-server/
> Friday, September 21st, 2007 at 5:49 am
> Joggame Server By eolaughlen
>
> Song Of The Day: Beautiful Day Artist: U2
>
> Today, I am pleased to announce a spin-off of the open source
> AOLserver project for devices: Joggame Server (pronounced jog-gu-mE).
>
> The technology, acquired by AOL in 1995 from NaviSoft, has a long
> history of technical achievement and innovation. It was one of the
> first to offer multi-threaded support. It was one of the first to
> embed an interpreter versus using the slower, and less efficient, CGI.
> For the iPhone it is one of the first web application development
> environments. Its ease-of-use, and long history of reliability,
> security, and cross-platform support, make it the perfect choice for
> the iPhone and other web-enabled devices.
>
> With this new vision my hope is that the Joggame Server project will
> continue to carry the legacy forward.
>
> ==
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to <[EMAIL 
> PROTECTED]> <[EMAIL PROTECTED]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
> field of your email blank.
>
>
> --
>
> Patrick O'Leary
>
> AOL Local Search Technologies
> Phone: + 1 703 265 8763
>
> You see, wire telegraph is a kind of a very, very long cat. You pull his tail 
> in New York and his head is meowing in Los Angeles.
>  Do you understand this?
> And radio operates exactly the same way: you send signals here, they receive 
> them there. The only difference is that there is no cat.
>   - Albert Einstein
>
> [image: View Patrick O Leary's LinkedIn profile]View Patrick O Leary's
> profile <http://www.linkedin.com/in/pjaol>
>
>
> --
> AOLserver - http://www.aolserver.com/
>
>
> To Remove yourself from this list, simply send an email to <[EMAIL 
> PROTECTED]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
> field of your email blank.
>
>


-- 
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.
<>

Re: [AOLSERVER] Large file uploads being limited

[Nathan Folkman]

Here's an example showing how to find what is currently configured, as well
as how to update the "maxupload" value. You can reset the value directly in
your AOLserver configuration file if you'd like.

Welcome to server1 running at /usr/local/aolserver/bin/nsd (pid 570)
AOLserver/4.5.0 (aolserver4_5) for osx built on May 11 2007 at 17:19:41
CVS Tag: $Name:  $

server1:nscp 1> ns_limits list
default

server1:nscp 2> ns_limits get default
nrunning 0 nwaiting 0 ntimeout 0 ndropped 0 noverflow 0 maxwait 100
maxupload 1024 timeout 60 maxrun 100

server1:nscp 3> ns_limits set default -maxupload [expr 1024 * 2]
nrunning 0 nwaiting 0 ntimeout 0 ndropped 0 noverflow 0 maxwait 100
maxupload 2048 timeout 60 maxrun 100

Hope that helps!

- n


On 8/10/07, Dossy Shiobara <[EMAIL PROTECTED]> wrote:
>
> On 2007.08.10, Pedro Liska <[EMAIL PROTECTED]> wrote:
> > I'm using AOLserver 4.5.0 with nsopenssl v3_0beta26 . When I post a file
> > that is greater than 10MB to the server, I get a "The connection was
> reset"
> > error in Mozilla and a "Internet Explorer cannot display the webpage"
> error
> > in IE7.
>
> Connection-oriented limits are now specified with the "ns_limit" command
> in 4.5.0 -- here's the snippet from the RELEASE file:
>
> ns_limit:
> The "ns_limit" command enables setting various resource
> limits for specified method/url combinations. These limits
> include such items as max concurrent connections, max file
> upload size, and timeouts waiting for connection processing.
> When limits are exceeded, connections are immediately
> dispatched to a dedicated "error" connection processing
> pool to generate a quick error response. By default all
> requests share the same default limits.  Coupled with the
> new "ns_pools" command, URL-based limits can provide for
> sophisticated resource management.
>
> -- Dossy
>
> --
> Dossy Shiobara  | [EMAIL PROTECTED] | http://dossy.org/
> Panoptic Computer Network   | http://panoptic.com/
>   "He realized the fastest way to change is to laugh at your own
> folly -- then you can let go and quickly move on." (p. 70)
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to <
> [EMAIL PROTECTED]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> Subject: field of your email blank.
>



-- 
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] Speaking of nsdci...

[Nathan Folkman]

Yeah I was planning on doing that, just wanted to finish up a few things
first. ;)

- n

On 8/9/07, Titi Alailima <[EMAIL PROTECTED]> wrote:
>
>  Back to the other discussion about community process, seems like some
> sort of mention on this list might get more input, and possibly help, for
> things in an "initial development release phase".  In depth discussion could
> go off-list, but just throwing out there that such a thing is being worked
> on would probably get a lot more involvement.  Putting things up on the wiki
> would also help (thanks to rcobb this one is up there as of last evening).
>
>
>
> Titi Ala'ilima
>
> Lead Architect
>
> MedTouch LLC
>
> 1100 Massachusetts Avenue
>
> Cambridge, MA 02138
>
> 617.621.8670 x309
>
>
>
> *From:* AOLserver Discussion [mailto:[EMAIL PROTECTED] *On
> Behalf Of *Nathan Folkman
> *Sent:* Wednesday, August 08, 2007 3:59 PM
> *To:* AOLSERVER@LISTSERV.AOL.COM
> *Subject:* Re: [AOLSERVER] Speaking of nsdci...
>
>
>
> It hasn't been "officially" released or announced as of yet. Tt's in more
> of an initial development release phase at the moment. Still lots of
> documentation to be done. There is currently no roadmap or timeline to share
> with everyone either at this time.
>
> - n
>
>
> --
> AOLserver - http://www.aolserver.com/
>
>
> To Remove yourself from this list, simply send an email to <[EMAIL 
> PROTECTED]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
> field of your email blank.
>
>


-- 
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] Speaking of nsdci...

[Nathan Folkman]

On 8/9/07, John Buckman <[EMAIL PROTECTED]> wrote:
>
> On Aug 9, 2007, at 2:07 AM, Nathan Folkman wrote:
>
> > A variant is used in production, although it's not exactly the same.
>
> Can you say more about variant?


The biggest difference is that the version in Google Code includes support
for using HTTP as a transport in addition to the persistent DCI_RPC
transport (see rpc.c).

> There's only a couple of us working on updating this code at the
> > moment, but I'd be more then happy to add anyone to the project who
> > is interested in contributing. There are a bunch of sample
> > configuration files, which honestly is the most confusing part of
> > this code base. Contact me off list if interested in pitching in,
> > and I'll get you set up.
>
> Ok, so it's messy right now, so probably not good for merging in.


It's actually not that bad - could always be better of course! ;)

Anyhow, I'm not that concerned about this feature right now, but
> Dossy mentioned wanting to know why I used lighthttpd for large files
> instead of aolserver, and that's how this async issue came up.


Not sure if this module would help you with large files, but AOLserver
4.5might given the changes made to how the driver thread interacts
with the
pool of connection threads. Suggest running some tests.

- n

-john
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to <
> [EMAIL PROTECTED]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> Subject: field of your email blank.
>



-- 
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] Speaking of nsdci...

[Nathan Folkman]

A variant is used in production, although it's not exactly the same.

There's only a couple of us working on updating this code at the moment, but
I'd be more then happy to add anyone to the project who is interested in
contributing. There are a bunch of sample configuration files, which
honestly is the most confusing part of this code base. Contact me off list
if interested in pitching in, and I'll get you set up.

- n

On 8/8/07, Jade Rubick <[EMAIL PROTECTED]> wrote:
>
> Is it used in production? Is there a plan to do any further development or
> documentation for it?
>
> Jade
>
> On 8/8/07, Nathan Folkman < [EMAIL PROTECTED]> wrote:
> >
> > It hasn't been "officially" released or announced as of yet. Tt's in
> > more of an initial development release phase at the moment. Still lots of
> > documentation to be done. There is currently no roadmap or timeline to share
> > with everyone either at this time.
> >
> > - n
> >
> > On 8/8/07, Rick Cobb < [EMAIL PROTECTED]> wrote:
> > >
> > >  I see it on Google Code ( http://code.google.com/p/nsdci/), but not
> > > on the modules wiki 
> > > (http://panoptic.com/wiki/aolserver/Modules#Miscellaneous
> > > ); I'll add it, but wondered why it wasn't there?  What's its status?
> > >
> > >
> > >
> > > -- ReC
> > >
> > >
> > > --
> > > AOLserver - http://www.aolserver.com/
> > >
> > >
> > > To Remove yourself from this list, simply send an email to <[EMAIL 
> > > PROTECTED]> with the
> > >
> > >
> > > body of "SIGNOFF AOLSERVER" in the email message. You can leave the 
> > > Subject: field of your email blank.
> > >
> > >
> >
> >
> > --
> > Nathan Folkman
> > [EMAIL PROTECTED]
> >
> >
> > --
> > AOLserver - http://www.aolserver.com/
> >
> >
> > To Remove yourself from this list, simply send an email to <[EMAIL 
> > PROTECTED]> with the
> >
> > body of "SIGNOFF AOLSERVER" in the email message. You can leave the 
> > Subject: field of your email blank.
> >
> >
>
>
> --
> Jade Rubick
> Senior Developer
> United eWay Volunteer Solutions
> [EMAIL PROTECTED]
> tel (503)285-4963
> fax (707)671-1333
>
> www.UNITEDeWAY.org
>
>
> --
> AOLserver - http://www.aolserver.com/
>
>
> To Remove yourself from this list, simply send an email to <[EMAIL 
> PROTECTED]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
> field of your email blank.
>
>


-- 
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] Speaking of nsdci...

[Nathan Folkman]

It hasn't been "officially" released or announced as of yet. Tt's in more of
an initial development release phase at the moment. Still lots of
documentation to be done. There is currently no roadmap or timeline to share
with everyone either at this time.

- n

On 8/8/07, Rick Cobb <[EMAIL PROTECTED]> wrote:
>
>  I see it on Google Code (http://code.google.com/p/nsdci/), but not on the
> modules wiki (http://panoptic.com/wiki/aolserver/Modules#Miscellaneous);
> I'll add it, but wondered why it wasn't there?  What's its status?
>
>
>
> -- ReC
>
>
> --
> AOLserver - http://www.aolserver.com/
>
>
> To Remove yourself from this list, simply send an email to <[EMAIL 
> PROTECTED]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
> field of your email blank.
>
>


-- 
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] aolserver focus

[Nathan Folkman]

It is possible. ;) There are a number of other things you can do as well to
help. One example is configuring your threads such that they die and get
reaped after a certain number of requests or time. This will cause memory
that is currently tied up in the thread's memory pool to be returned back to
the shared memory pool, which can have the net effect of lowering the
overall process size.

- n

On 8/7/07, Daniël Mantione <[EMAIL PROTECTED]> wrote:
>
>
>
> Op Tue, 7 Aug 2007, schreef Nathan Folkman:
>
> > You might also want to try running AOLserver without the Tcl threaded
> > allocator (Zippy). You might want to try Hoard or if on Linux maybe give
> > Google's TCMalloc a shot. Remember, the "Zippy" allocator is optimized
> for
> > lock avoidance, and this comes at the cost of greater memory overhead.
>
> Yup, I have installations where AOLserver uses very little memory. Try
> this with Apache :)
>
> wwwrun4552  0.0  1.4   7380  3612 ?Sl   Jul26   2:21
> /opt/aolserver/bin/nsd -ft /opt/aolserver/nsd.tcl -u wwwrun
>
> Daniël
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to <
> [EMAIL PROTECTED]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> Subject: field of your email blank.




-- 
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] aolserver focus

[Nathan Folkman]

s implicit (i.e. give a command line option so each user can
> > automatically have his own config file, tcl library, etc.), and
> installing
> > AOLserver on a server rather than Apache becomes feasible for a hosting
> > provider.
> >
> > > However the server tends to
> > > grow in memory size over time and running multiple independent servers
> > > just worsens the problem.
> >
> > I restart my AOLserver at 04:00 each night, which is enough to
> > elmininate the problem, but this is indeed an issue for current users. I
> > believe it has little to do with popularity, though.
> >
> > Daniël
> >
> >
> > --
> > AOLserver - http://www.aolserver.com/
> >
> > To Remove yourself from this list, simply send an email to
> > <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the
> > email message. You can leave the Subject: field of your email blank.
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to <
> [EMAIL PROTECTED]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> Subject: field of your email blank.
>



-- 
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command

[Nathan Folkman]

We actually also tried a similar process a few years ago, but it failed due
to various conflicts and general lack of interest. Would be great to get
something like that going again.

- n

On 8/3/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
>
> > The reason why I reach out to organizations privately is because some of
> > them are building commercial products on top of AOLserver and may not be
> > comfortable discussing their needs in a public forum.
>
> Well, the way we handle such situations in the openacs community is that
> changes go through an approval process which we've copied from the Tcl
> crew.
>
> So these things CAN'T happen in private.
>
> The motivation - or the identity of a paying customer - can be kept
> private, of course, but changes in core code that changes functionality
> must be approved by core developers in a PUBLIC PROCESS.
>
> The developer community around AOLserver is small, so informality's fine
> IMO, but transparency is crucial.
>
> And BTW, I recognize that you've communicated with the OpenACS community
> as well as various companies, and that's a good thing.
>
> > Don, what would you change to improve things?  Of those suggestions,
> > which could you personally do to help the project?
>
> Unfortunately my volunteer time is still allocated to OpenACS, so I can't
> really offer to help on a sustained basis, though I did drop in and
> implement virtual server awareness in ns_cache back when virtual servers
> were put in.
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to <
> [EMAIL PROTECTED]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> Subject: field of your email blank.
>



-- 
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command

[Nathan Folkman]

I'm someone who can, and will, remove your subscription from this list.

- n

On 8/3/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
>
> > Please watch the vulgar language - there's simply no need for it.
>
> And precisely who are you to say so?
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to <
> [EMAIL PROTECTED]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> Subject: field of your email blank.
>



-- 
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command

[Nathan Folkman]

Please watch the vulgar language - there's simply no need for it. Thanks!

- n

On 8/3/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
>
> > On 2007.08.02, Tom Jackson <[EMAIL PROTECTED]> wrote:
> >Should I step down as project
> > leader and let someone else take over?  The irony is that the title
> > really doesn't mean much at all.
>
> I suspect that you simply don't understand what it means in the context of
> a community-based open source project.
>
> It doesn't mean that you arbitrarily make decisions quietly and then go
> implement them without discussion with the community.
>
> Please share with us YOUR definition!
>
> > So, in the spirit of open source software meritocracies: please place
> > your money where your mouth is.  Come up with a list of actionable
> > changes you'd make if you were king.  Lets hear it--and if everyone
> > agrees to a particular change, we'll declare it made.  (Note: declaring
> > anything to a volunteer-driven organization doesn't guarantee that
> > anyone will actually do it.)
>
> The graceful thing to do would be to say "I fucked up, sorry, there was no
> malice on my end, it was a simple mistake and I take responsibility for
> not communicating my ideas to the community before implementing them.
> Let's move on!"
>
> But your use of the word "king" makes me think you don't really quite
> understand the role, unless you're thinking of a king working in a
> parliamentary government.
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to <
> [EMAIL PROTECTED]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> Subject: field of your email blank.
>



-- 
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] learning from naviserver

[Nathan Folkman]

There's a lot of really innovative stuff going on with the naviserver
project, and they've done a lot of work to stay in sync with changes being
made to the core AOLserver project. I suggest folks check it out.

- n

On 8/3/07, John Buckman <[EMAIL PROTECTED]> wrote:
>
> > So, in the spirit of open source software meritocracies: please place
> > your money where your mouth is.  Come up with a list of actionable
> > changes you'd make if you were king.  Lets hear it--and if everyone
> > agrees to a particular change, we'll declare it made.  (Note:
> > declaring
> > anything to a volunteer-driven organization doesn't guarantee that
> > anyone will actually do it.)
>
> In no way do I want to be king, and in an effort to calm things down
> a bit, let me say that I'm really, really happy with aolserver, and
> the last major release had amazing things in it. My only real nit is
> that so many cool things are not-so-well-documented, but hell, it's
> open source. I just spot things in the text files, and then figure it
> out by reading the C code, which is often commented, but always clean
> and readable.
>
> On the subject of cool things and not-so-well-documented, I'd like to
> bring up Naviserver (you can find it on sourceforge, it's an
> independent fork of aolsever, at http://naviserver.sourceforge.net).
> Their fork of aolserver has an insane number of changes to it, and
> lots of great ideas (look at http://naviserver.cvs.sf.net/naviserver/
> naviserver/ChangeLog?view=markup).  There's a handful of developers
> working on it, and it seems like a real hotbed of innovation.
>
> I've not used naviserver, though I evaluated it seriously, because it
> has so many differences to aolserver, almost all undocumented, that
> it was really hard for me to get up to speed to, and I found it less
> reliable than aolserver, probably because of all those innovations.
> I kind of like the slower pace of aolserver, I can actually run a
> production web site on it. This is not meant as an insult to Vlad and
> the other Naviserver developers, I'm just pointing out how the two
> development communities differ.
>
> However, I was wondering if we should perhaps look at merging back
> some of the best things in naviserver, back into aolserver.  In fact,
> maybe we should treat the aolserver/naviserver split like ubuntu
> treats its two releases, and recognize that naviserver as an
> innovative, highly chaotic playground, and merging back in the best
> ideas from it back into aolserver after a long delay (6 months to a
> year) once each feature has settled down a bit and we can evaluate
> whether, in hindsight, it really was a good idea and the way it was
> implemented turned out well.
>
> I'd be game to go through the naviserver changelog in the future, and
> be part of a discussion of what is in there that we might want to
> merge back into aolserver.
>
> -john
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to <
> [EMAIL PROTECTED]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> Subject: field of your email blank.
>



-- 
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command

[Nathan Folkman]

There was a desire to be able to tune some of the server parameters while
the server was running in order to avoid certain cases where a restart would
result in the loss of various caches which had already been warmed up.
Admittedly this requirement came from requirements for various AOL
properties, but most of the new features in 4.5 came about this way.

- n

On 8/2/07, Dossy Shiobara <[EMAIL PROTECTED]> wrote:
>
> On 2007.08.01, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
> > One of the hallmarks of AOLserver has always been ease of configuration.
> > My sense is that we're drifting away from that?
>
> Frankly, one of the biggest FAQ's is "how do I tune AOLserver?"  IMHO,
> "ease of configuration" is "less to manually tune" which means settings
> need to be able to be adjusted at runtime.
>
> Seriously, what value does being able to set max connections manually
> really offer?  What people really want is a server that uses resources
> efficiently but in a way which doesn't run itself out of resources and
> starve itself.  Rarely do I actually hear people say "yes, I really want
> to limit the number of connections" but rather "I want to allow as many
> connections as my hardware can handle--how high should I set
> max connections?"
>
> AOLserver is going to become even easier to set up and configure.  These
> are growing pains, unfortunately.
>
> -- Dossy
>
> --
> Dossy Shiobara  | [EMAIL PROTECTED] | http://dossy.org/
> Panoptic Computer Network   | http://panoptic.com/
>   "He realized the fastest way to change is to laugh at your own
> folly -- then you can let go and quickly move on." (p. 70)
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to <
> [EMAIL PROTECTED]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> Subject: field of your email blank.
>



-- 
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command

[Nathan Folkman]

On 8/2/07, Andrew Piskorski <[EMAIL PROTECTED]> wrote:
>
> On Thu, Aug 02, 2007 at 11:51:07AM -0400, Nathan Folkman wrote:
> > There was a lot of discussion internally about whether or not to support
> > virtual servers going forward.
>
> Uh, was there some good reason that discussion was not carried out on
> this AOLserver email list?  Was there in fact any reason at all, even
> a bad one?


Not that I can remember. I think a lot of the work was actually done while
Jim was on vacation. ;)

--
> Andrew Piskorski <[EMAIL PROTECTED]>
> http://www.piskorski.com/
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to <
> [EMAIL PROTECTED]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> Subject: field of your email blank.
>



-- 
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command

[Nathan Folkman]

There was a lot of discussion internally about whether or not to support
virtual servers going forward. The main impetus was to try and further
simplify the code base as much as possible. Virtual servers, while useful to
many, add a great deal of complexity to the code.

- n

On 8/2/07, Tom Jackson <[EMAIL PROTECTED]> wrote:
>
> On Wednesday 01 August 2007 23:53, [EMAIL PROTECTED] wrote:
> > > Maybe this is the intent, although it would be easy to add the
> > > server
> > > name to the key.
> >
> > ISTM that virtual servers weren't considered when this code was written
> ...
> >
> > All code in current versions should be fully aware of virtual servers.
>
> The previous code in 4.0.10 was in urlspace.c and queue.c and was fully
> virtual server aware. The current version simply ditches this for no good
> reason, and by that I mean that the map storage and search are fully
> virtual
> server aware, but the threadpool creation code is a simple hash table
> keyed
> to the name of the threadpool.
>
> It also appears that on startup, two threadpools are created 'default'
> and 'error'. These are shared across virtual servers. Any redefinition of
> these affects all virtual servers.
>
> So question: we now have interps separate from threads, but what about
> memory,
> code, etc. Is this part of the interp or does it hang out with the thread?
> If
> the thread is just a clean slate, at least it is safe to reuse in another
> virtual server, otherwise this is a big bend over routine. But then we are
> left with the other option: threads pick up interps and run anywhere and
> everywhere on the virtual server and the interp eventually grows in size.
> Even if threads exit with given timeout,  do interps? This negates the
> reason
> for threadpools entirely.
>
> This really needs to be discussed because the programming model could be
> disasterously affected and complicated by these decisions.
>
> tom jackson
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to <
> [EMAIL PROTECTED]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> Subject: field of your email blank.
>



-- 
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command

[Nathan Folkman]

_pools get $poolName]"
> set poolConfigSize [ns_set size $poolConfigSet]
> for {set j 0} {$j < $poolConfigSize} {incr j} {
> if {[string tolower [ns_set key $poolConfigSet $j]] eq "map"}
> {
>     set mapList [split [ns_set value $poolConfigSet $j]]
> set poolMethod [lindex $mapList 0]
> set poolPattern [lindex $mapList 1]
> ns_pools register $poolName [ns_info server] $poolMethod
> $poolPattern
> ns_log Notice "ns_pools registered $poolName [ns_info
> server] $poolMethod $poolPattern"
> }
> }
> }
> }
>
> -
>
> tom jackson
>
> On Wednesday 01 August 2007 10:50, Nathan Folkman wrote:
> > Bingo - Michael Andrews just contributed such a script.
> >
> > - n
> >
> > On 8/1/07, Tom Jackson <[EMAIL PROTECTED]> wrote:
> > > I would just like to point out that 4.5 was released over one year
> ago,
> > > so the
> > > cat is already out of the bag.
> > >
> > > But more important: it seems like this is not really a problem. The
> old
> > > configuration can still be used. All we need is a script which reads
> the
> > > configuration data and runs the new commands.
> > >
> > > Is there a command reference anywhere, or could someone provide a
> simple
> > > example of use?
> > >
> > > If so, I can write a script and try to figure out where it should go
> so
> > > that
> > > it executes at the correct time. When is this time anyway? Apparently
> it
> > > is
> > > not during the sourcing of the config file.
> > >
> > > tom jackson
> > >
> > > On Wednesday 01 August 2007 09:33, Nathan Folkman wrote:
> > > > As I said before, if you have issues with the changes made in 4.5,
> > > > simply do not upgrade. There were a number of factors that led to
> our
> > > > decision to release as we did, when we did. Unfortunately a lot of
> the
> > > > backwards compatibility work never got completed. Sorry.
> > >
> > > --
> > > AOLserver - http://www.aolserver.com/
> > >
> > > To Remove yourself from this list, simply send an email to <
> > > [EMAIL PROTECTED]> with the
> > > body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> > > Subject: field of your email blank.
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to <
> [EMAIL PROTECTED]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> Subject: field of your email blank.
>



-- 
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command

[Nathan Folkman]

Bingo - Michael Andrews just contributed such a script.

- n

On 8/1/07, Tom Jackson <[EMAIL PROTECTED]> wrote:
>
> I would just like to point out that 4.5 was released over one year ago, so
> the
> cat is already out of the bag.
>
> But more important: it seems like this is not really a problem. The old
> configuration can still be used. All we need is a script which reads the
> configuration data and runs the new commands.
>
> Is there a command reference anywhere, or could someone provide a simple
> example of use?
>
> If so, I can write a script and try to figure out where it should go so
> that
> it executes at the correct time. When is this time anyway? Apparently it
> is
> not during the sourcing of the config file.
>
> tom jackson
>
> On Wednesday 01 August 2007 09:33, Nathan Folkman wrote:
> > As I said before, if you have issues with the changes made in 4.5,
> > simply do not upgrade. There were a number of factors that led to our
> > decision to release as we did, when we did. Unfortunately a lot of the
> > backwards compatibility work never got completed. Sorry.
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to <
> [EMAIL PROTECTED]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> Subject: field of your email blank.
>



-- 
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command

[Nathan Folkman]

I think we did actually talk about some of the stuff that was coming down
the pipe when we used to have those AOLserver chats. Not sure what happened
to those. I personally haven't been "officially" assigned to AOLserver
support for a while now.

- n

On 8/1/07, Nathan Folkman <[EMAIL PROTECTED]> wrote:
>
> To be honest and blunt, the community process in terms of AOLserver is
> rather broken, and has been for some time. The 4.5 work was done in-house
> here at AOL, and with limited community involvement. A lot of the
> "experimental" features were added to deal with specific scaling challenges
> we knew we would be facing here.
>
> When decided to go ahead and release 4.5, the thought was that we would
> eventually finish up any backwards compatibility work that needed to be
> done. We also assumed that if there were issues, people would simply hold
> off on upgrading, or work around the issues.
>
> Would documentation have helped - of course. Would a roadmap be useful -
> definitely. Agree with everything you, and others are saying, but we're not
> there yet. Sorry.
>
> - n
>
> On 8/1/07, Rick Cobb <[EMAIL PROTECTED]> wrote:
> >
> > I mostly sympathize with your sentiments. Incompatible changes should be
> > documented.  And design documents & directions for AOLServer belong in
> > public on the AOLServer wiki, not hidden.
> >
> > I.e., if when someone decided this was a good idea, there was an
> > "experimental features" page for 4.5 that outlined the idea, that would
> > at least form the basis for (a) docs, and (b) community discussion.  If
> > someone (like you or me) needed that backward compatibility, we could
> > contribute it.  At present, it still seems like the docs we see on the
> > wiki are either generated from long-ago internal documentation, or not
> > written at all.
> >
> > And as for roadmaps:
> > "Your search - roadmap - did not match any documents."
> >
> > However, I would like to point out that one of the real problems with
> > the AOLServer community for a while was its perceived inability to pick
> > up innovations. Discouraging change from inside AOL is only going to
> > make the problem worse.  We should be encouraging experimentation -- but
> > with community feedback.
> >
> > -- ReC
> >
> >
> > -Original Message-
> > From: AOLserver Discussion [mailto: [EMAIL PROTECTED] On
> > Behalf
> > Of [EMAIL PROTECTED]
> > Sent: Wednesday, August 01, 2007 8:57 AM
> > To: AOLSERVER@LISTSERV.AOL.COM
> > Subject: Re: [AOLSERVER] configured minthreads, maxthreads doesnt show
> > up with [ns_server threads] command
> >
> > > Technically everything in the configuration file is a Tcl command
> > > (ns_section, ns_param, etc.) so it's really not that much of a
> > stretch.
> > > But
> > > I agree, it is different.
> >
> > Technically, the configuration file is a bucket of bits, but that's not
> > a
> > very useful observation.
> >
> > > Had we ever actually gotten around to deploying AOLserver 4.5
> > internally
> > > here at AOL, I'm sure we would have at least done the work to provide
> > some
> > > backwards compatibility wrappers for things like this, but as yet we
> > > haven't.
> >
> > Geezus.  In this case, the code simply shouldn't've gone in.
> >
> > Or it should've been offered to the community in its incomplete form,
> > with
> > the offer to incorporate it in 4.5 if volunteers in the community
> > completed it.
> >
> >
> > --
> > AOLserver - http://www.aolserver.com/
> >
> > To Remove yourself from this list, simply send an email to
> > <[EMAIL PROTECTED]> with the
> > body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> > Subject: field of your email blank.
> >
> >
> > --
> > AOLserver - http://www.aolserver.com/
> >
> > To Remove yourself from this list, simply send an email to <[EMAIL 
> > PROTECTED]>
> > with the
> > body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> > Subject: field of your email blank.
> >
>
>
>
> --
> Nathan Folkman
> [EMAIL PROTECTED]
>



-- 
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command

[Nathan Folkman]

To be honest and blunt, the community process in terms of AOLserver is
rather broken, and has been for some time. The 4.5 work was done in-house
here at AOL, and with limited community involvement. A lot of the
"experimental" features were added to deal with specific scaling challenges
we knew we would be facing here.

When decided to go ahead and release 4.5, the thought was that we would
eventually finish up any backwards compatibility work that needed to be
done. We also assumed that if there were issues, people would simply hold
off on upgrading, or work around the issues.

Would documentation have helped - of course. Would a roadmap be useful -
definitely. Agree with everything you, and others are saying, but we're not
there yet. Sorry.

- n

On 8/1/07, Rick Cobb <[EMAIL PROTECTED]> wrote:
>
> I mostly sympathize with your sentiments. Incompatible changes should be
> documented.  And design documents & directions for AOLServer belong in
> public on the AOLServer wiki, not hidden.
>
> I.e., if when someone decided this was a good idea, there was an
> "experimental features" page for 4.5 that outlined the idea, that would
> at least form the basis for (a) docs, and (b) community discussion.  If
> someone (like you or me) needed that backward compatibility, we could
> contribute it.  At present, it still seems like the docs we see on the
> wiki are either generated from long-ago internal documentation, or not
> written at all.
>
> And as for roadmaps:
> "Your search - roadmap - did not match any documents."
>
> However, I would like to point out that one of the real problems with
> the AOLServer community for a while was its perceived inability to pick
> up innovations. Discouraging change from inside AOL is only going to
> make the problem worse.  We should be encouraging experimentation -- but
> with community feedback.
>
> -- ReC
>
>
> -Original Message-
> From: AOLserver Discussion [mailto:[EMAIL PROTECTED] On Behalf
> Of [EMAIL PROTECTED]
> Sent: Wednesday, August 01, 2007 8:57 AM
> To: AOLSERVER@LISTSERV.AOL.COM
> Subject: Re: [AOLSERVER] configured minthreads, maxthreads doesnt show
> up with [ns_server threads] command
>
> > Technically everything in the configuration file is a Tcl command
> > (ns_section, ns_param, etc.) so it's really not that much of a
> stretch.
> > But
> > I agree, it is different.
>
> Technically, the configuration file is a bucket of bits, but that's not
> a
> very useful observation.
>
> > Had we ever actually gotten around to deploying AOLserver 4.5
> internally
> > here at AOL, I'm sure we would have at least done the work to provide
> some
> > backwards compatibility wrappers for things like this, but as yet we
> > haven't.
>
> Geezus.  In this case, the code simply shouldn't've gone in.
>
> Or it should've been offered to the community in its incomplete form,
> with
> the offer to incorporate it in 4.5 if volunteers in the community
> completed it.
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to
> <[EMAIL PROTECTED]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> Subject: field of your email blank.
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to <
> [EMAIL PROTECTED]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> Subject: field of your email blank.
>



-- 
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command

[Nathan Folkman]

As I said before, if you have issues with the changes made in 4.5,
simply do not upgrade. There were a number of factors that led to our
decision to release as we did, when we did. Unfortunately a lot of the
backwards compatibility work never got completed. Sorry.


On 8/1/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
> > Technically everything in the configuration file is a Tcl command
> > (ns_section, ns_param, etc.) so it's really not that much of a stretch.
> > But
> > I agree, it is different.
>
> Technically, the configuration file is a bucket of bits, but that's not a
> very useful observation.
>
> > Had we ever actually gotten around to deploying AOLserver 4.5 internally
> > here at AOL, I'm sure we would have at least done the work to provide some
> > backwards compatibility wrappers for things like this, but as yet we
> > haven't.
>
> Geezus.  In this case, the code simply shouldn't've gone in.
>
> Or it should've been offered to the community in its incomplete form, with
> the offer to incorporate it in 4.5 if volunteers in the community
> completed it.
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to
> <[EMAIL PROTECTED]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject:
> field of your email blank.
>


-- 
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command

[Nathan Folkman]

Yep, the documentationn or lack there of, continues to dog us. Did you
read the release notes at least? It doesn't specifically mention this
incompatibility, but does contain a lot of useful information.


On 8/1/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
> > You are absolutely correct. Lack of documentation continues to be one of
> > the
> > biggest issues with this project in my opinion. Not sure how best to
> > resolve
> > this at this point to be completely honest. Would be a great start if
> > other
> > folks would start contributing to the example configuration files and the
> > man pages. Is there anything that's making this difficult, and preventing
> > people from helping out?
>
> So, if I'm not mistaken, not only were existing config files broken, the
> basic philosophy of how AOLserver is configured changed for this important
> case (dynamic proc call), but ...
>
> There's not even any documentation telling people this?
>
> Something this significant shouldn't be implemented without documentation,
> and frankly, it's the developer's responsibility to make sure it's done.
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to
> <[EMAIL PROTECTED]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject:
> field of your email blank.
>


-- 
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command

[Nathan Folkman]

Simple answer - don't upgrade to 4.5. As I was trying to explain
before, we knew some things in 4.5 would not be backwards compatible.


On 8/1/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
> > Nathan,
> >
> > This has to be the most bizzar change to the configuration setup for
> > AOLserver, is it really true? Now you have to execute commands inside the
> > config file to set this?
>
> This is absolutely crazy.  The init file has never required dynamic
> execution of procs to work, and to have something this basic as an
> exception is nuts.
>
> Not to mention that breaking every bloody site's config file in the
> process is  rather rude.
>
> Some of us still use AOLserver for production sites, hard as that is to
> believe.
>
> Was there community discussion of this change before it was implemented?
> It represents a huge change in configuration philosophy.
>
> Who's bright idea was this?
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to
> <[EMAIL PROTECTED]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject:
> field of your email blank.
>


-- 
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command

[Nathan Folkman]

What about just contributing documentation? What is making that hard?


On 8/1/07, Michael Andrews <[EMAIL PROTECTED]> wrote:
>
> I think lack of process is a big deterrent.  How do changes get
> rolled into releases, what are the coding standards, is there a
> review process, etc.
>
>
>
> On Aug 1, 2007, at 10:29 AM, Nathan Folkman wrote:
>
> > You are absolutely correct. Lack of documentation continues to be
> > one of the biggest issues with this project in my opinion. Not sure
> > how best to resolve this at this point to be completely honest.
> > Would be a great start if other folks would start contributing to
> > the example configuration files and the man pages. Is there
> > anything that's making this difficult, and preventing people from
> > helping out?
> >
> > - n
> >
> > On 8/1/07, Ian Harding <[EMAIL PROTECTED]> wrote:
> > Better I think to add them (commented out if need be) the default
> > config file that ships with 4.5.  I searched the documentation and
> > couldn't find any mention of the new setup (until someone pointed it
> > out) but plenty of talk about the old ones.  I think it's in the
> > release notes, but nobody reads those ;^)
> >
> > - Ian
> >
> > On 7/30/07, Jeff Rogers <[EMAIL PROTECTED]> wrote:
> > > Nathan Folkman wrote:
> > > > Those parameters moved and are now controlled via the
> > "ns_pools" Tcl
> > > > command:
> > >
> > > This issue keeps coming up.  How hard would it be to add in (for the
> > > next point release) compatibility code in init.tcl to parse the
> > > old-style parameters?
> > >
> > > -J
> > >
> > >
> > > --
> > > AOLserver - http://www.aolserver.com/
> > >
> > > To Remove yourself from this list, simply send an email to <
> > [EMAIL PROTECTED]> with the
> > > body of "SIGNOFF AOLSERVER" in the email message. You can leave
> > the Subject: field of your email blank.
> > >
> >
> >
> > --
> > AOLserver - http://www.aolserver.com/
> >
> > To Remove yourself from this list, simply send an email to
> > <[EMAIL PROTECTED]> with the
> > body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> > Subject: field of your email blank.
> >
> >
> >
> > --
> > Nathan Folkman
> > [EMAIL PROTECTED]
> >
> > --
> > AOLserver - http://www.aolserver.com/
> >
> >
> > To Remove yourself from this list, simply send an email to
> > <[EMAIL PROTECTED]> with the
> > body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> > Subject: field of your email blank.
>
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to
> <[EMAIL PROTECTED]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject:
> field of your email blank.
>


-- 
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command

[Nathan Folkman]

Yes and no. ;)

Technically everything in the configuration file is a Tcl command
(ns_section, ns_param, etc.) so it's really not that much of a stretch. But
I agree, it is different.

Here's the deal, a decision was made a while back to try some new things
with AOLserver 4.5, some of which we knew wouldn't be backwards compatible.
Thread pools and limits are two pretty radical changes that were made to the
core.

Had we ever actually gotten around to deploying AOLserver 4.5 internally
here at AOL, I'm sure we would have at least done the work to provide some
backwards compatibility wrappers for things like this, but as yet we
haven't.

I'll try and update the default configuration file with some examples soon.
Sorry for the confusion.

- n



On 8/1/07, Tom Jackson <[EMAIL PROTECTED]> wrote:
>
> Nathan,
>
> This has to be the most bizzar change to the configuration setup for
> AOLserver, is it really true? Now you have to execute commands inside the
> config file to set this?
>
> Since some have called for examples, would it be possible for the author
> of
> these changes to provide a few. I have been using the threadpool
> configuration as shown in this example (start at server.tcl, then source
> threadpool files):
>
> From: <http://rmadilo.com/m2/servers/rmadilo/config/>
> # threadpool-default.tcl
> # Single Threadpool
>
> # pool = default
>
> ns_section "ns/server/${server}/pool/$pool"
> ns_param   maxconnections100
> ns_param   minthreads4 ;# 0
> ns_param   maxthreads10
> ns_param   threadtimeout 120
> ns_param   map   "GET /"
> ns_param   map   "POST /"
>
>
> tom jackson
>
> On Thursday 26 July 2007 08:53, Nathan Folkman wrote:
> > You'd actually want to do it by adding the following to the end of your
> > configuration file:
> >
> > ns_pools set procsmsgmgr -maxconns 100 -maxthreads 20 -minthreads 10
> > -timeout 10
> > ns_pools register procsmsgmgr server1 POST /proc/msgmgr
> >
> > You can then verify that everything worked via the AOLserver control
> port:
> >
> > server1:nscp 1> ns_pools list
> > procsmsgmgr default error
> >
> > server1:nscp 2> ns_server threads procsmsgmgr
> > {min 10} {max 20} {current 10} {idle 10} {stopping 0}
> >
> > server1:nscp 3> ns_pools get procsmsgmgr
> > minthreads 10 maxthreads 20 idle 10 current 10 maxconns 100 queued 0
> > timeout 10
> >
> > Hope that helps!
> >
> > - n
> >
> > On 7/26/07, Shedi Shedi <[EMAIL PROTECTED]> wrote:
> > > Thanks Nathan for the info. I'm trying to configure a pool but
> ns_server
> > > threads procmsgmgr returns error on server log.
> > >
> > > [26/Jul/2007:18:06:07][4968.3074280352][-conn:0-] Error: Tcl
> exception:
> > > no such pool: procmsgmgr
> > > while executing
> > > "ns_server threads procmsgmgr"
> > >
> > > #
> > > ns_section ns/server/${servername}/pools
> > > ns_param  procmsgmgr "Message Manager Receiving Pool"
> > >
> > > ns_section ns/server/${servername}/pool/procmsgmgr
> > > ns_param map {POST /proc/msgmgr}
> > > ns_param maxconnections   100
> > > ns_param maxdropped   0
> > > ns_param maxthreads   20
> > > ns_param minthreads   10
> > > ns_param threadtimeout60
> > >
> > > Can you tell if my configuration is correct?
> > >
> > > regards,
> > >
> > > On 7/26/07, Nathan Folkman <[EMAIL PROTECTED]> wrote:
> > > > Those parameters moved and are now controlled via the "ns_pools" Tcl
> > > > command:
> > > >
> > > > ns_pools:
> > > > The "ns_pools" command enables configuration of one or more
> > > > pools of connection processing threads. The pools allow
> > > > certain requests to be handled by specific threads. This
> > > > could, for example, ensure multiple long running requests
> > > > don't block other short running requests.  Pools are
> selected
> > > > based on method/url pairs similar to the mappings managed
> > > > by the "ns_register_proc" command.  By default, all requests
> > > > are handled by a single, unlimited, "default" pool.  There
> > > > is also an "error" pool as described below.  Coupled with
> > > > the new "

Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command

[Nathan Folkman]

You are absolutely correct. Lack of documentation continues to be one of the
biggest issues with this project in my opinion. Not sure how best to resolve
this at this point to be completely honest. Would be a great start if other
folks would start contributing to the example configuration files and the
man pages. Is there anything that's making this difficult, and preventing
people from helping out?

- n

On 8/1/07, Ian Harding <[EMAIL PROTECTED]> wrote:
>
> Better I think to add them (commented out if need be) the default
> config file that ships with 4.5.  I searched the documentation and
> couldn't find any mention of the new setup (until someone pointed it
> out) but plenty of talk about the old ones.  I think it's in the
> release notes, but nobody reads those ;^)
>
> - Ian
>
> On 7/30/07, Jeff Rogers <[EMAIL PROTECTED]> wrote:
> > Nathan Folkman wrote:
> > > Those parameters moved and are now controlled via the "ns_pools" Tcl
> > > command:
> >
> > This issue keeps coming up.  How hard would it be to add in (for the
> > next point release) compatibility code in init.tcl to parse the
> > old-style parameters?
> >
> > -J
> >
> >
> > --
> > AOLserver - http://www.aolserver.com/
> >
> > To Remove yourself from this list, simply send an email to <
> [EMAIL PROTECTED]> with the
> > body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> Subject: field of your email blank.
> >
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to <
> [EMAIL PROTECTED]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> Subject: field of your email blank.
>



-- 
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command

[Nathan Folkman]

Probably would be that difficult, although honestly not a priority for
anyone here. Have at it! ;)

- n

On 7/30/07, Jeff Rogers <[EMAIL PROTECTED]> wrote:
>
> Nathan Folkman wrote:
> > Those parameters moved and are now controlled via the "ns_pools" Tcl
> > command:
>
> This issue keeps coming up.  How hard would it be to add in (for the
> next point release) compatibility code in init.tcl to parse the
> old-style parameters?
>
> -J
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to <
> [EMAIL PROTECTED]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> Subject: field of your email blank.
>



-- 
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command

[Nathan Folkman]

You'd actually want to do it by adding the following to the end of your
configuration file:

ns_pools set procsmsgmgr -maxconns 100 -maxthreads 20 -minthreads 10
-timeout 10
ns_pools register procsmsgmgr server1 POST /proc/msgmgr

You can then verify that everything worked via the AOLserver control port:

server1:nscp 1> ns_pools list
procsmsgmgr default error

server1:nscp 2> ns_server threads procsmsgmgr
{min 10} {max 20} {current 10} {idle 10} {stopping 0}

server1:nscp 3> ns_pools get procsmsgmgr
minthreads 10 maxthreads 20 idle 10 current 10 maxconns 100 queued 0
timeout 10

Hope that helps!

- n

On 7/26/07, Shedi Shedi <[EMAIL PROTECTED]> wrote:
>
> Thanks Nathan for the info. I'm trying to configure a pool but ns_server
> threads procmsgmgr returns error on server log.
>
> [26/Jul/2007:18:06:07][4968.3074280352][-conn:0-] Error: Tcl exception:
> no such pool: procmsgmgr
> while executing
> "ns_server threads procmsgmgr"
>
> #
> ns_section ns/server/${servername}/pools
> ns_param  procmsgmgr "Message Manager Receiving Pool"
>
> ns_section ns/server/${servername}/pool/procmsgmgr
> ns_param map {POST /proc/msgmgr}
> ns_param maxconnections   100
> ns_param maxdropped   0
> ns_param maxthreads   20
> ns_param minthreads   10
> ns_param threadtimeout60
>
> Can you tell if my configuration is correct?
>
> regards,
>
> On 7/26/07, Nathan Folkman <[EMAIL PROTECTED]> wrote:
> >
> > Those parameters moved and are now controlled via the "ns_pools" Tcl
> > command:
> >
> > ns_pools:
> > The "ns_pools" command enables configuration of one or more
> > pools of connection processing threads. The pools allow
> > certain requests to be handled by specific threads. This
> > could, for example, ensure multiple long running requests
> > don't block other short running requests.  Pools are selected
> > based on method/url pairs similar to the mappings managed
> > by the "ns_register_proc" command.  By default, all requests
> > are handled by a single, unlimited, "default" pool.  There
> > is also an "error" pool as described below.  Coupled with
> > the new "ns_limits" command, pools can provide for sophisticated
> > resource management.
> >
> > See also the new "ns_limits" command:
> >
> > ns_limit:
> > The "ns_limit" command enables setting various resource
> > limits for specified method/url combinations. These limits
> > include such items as max concurrent connections, max file
> > upload size, and timeouts waiting for connection processing.
> > When limits are exceeded, connections are immediately
> > dispatched to a dedicated "error" connection processing
> > pool to generate a quick error response. By default all
> > requests share the same default limits.  Coupled with the
> > new "ns_pools" command, URL-based limits can provide for
> > sophisticated resource management.
> >
> > On 7/26/07, Shedi Shedi < [EMAIL PROTECTED]> wrote:
> > >
> > > Its 4.5.0
> > >
> > >
> > > On 7/26/07, Nathan Folkman <[EMAIL PROTECTED] > wrote:
> > > >
> > > > What version of AOLserver are you running?
> > > >
> > > > On 7/26/07, Shedi Shedi < [EMAIL PROTECTED]> wrote:
> > > >
> > > > > Hi
> > > > >
> > > > > Patform: suse 10.1 (2.6.16.21-0.25-default)
> > > > >
> > > > > I have configured my nsd config.tcl as below:
> > > > >
> > > > > ns_param   maxconnections  100   ;# Max connections to put on
> > > > > queue
> > > > > ns_param   maxdropped  0 ;# Shut down if dropping too
> > > > > many conns
> > > > > ns_param   maxthreads  50;# Tune this to scale your
> > > > > server
> > > > > ns_param   minthreads  20 ;# Tune this to scale your
> > > > > server
> > > > > ns_param   threadtimeout   30   ;# Idle threads die at this
> > > > > rate
> > > > >
> > > > >
> > > > > ns_server threads command shows the min as zero, max as 10.
> > > > >
> > > > > Server Threads
> > > > > min0
> > > > > max

Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command

[Nathan Folkman]

Those parameters moved and are now controlled via the "ns_pools" Tcl
command:

ns_pools:
The "ns_pools" command enables configuration of one or more
pools of connection processing threads. The pools allow
certain requests to be handled by specific threads. This
could, for example, ensure multiple long running requests
don't block other short running requests.  Pools are selected
based on method/url pairs similar to the mappings managed
by the "ns_register_proc" command.  By default, all requests
are handled by a single, unlimited, "default" pool.  There
is also an "error" pool as described below.  Coupled with
the new "ns_limits" command, pools can provide for sophisticated
resource management.

See also the new "ns_limits" command:

ns_limit:
The "ns_limit" command enables setting various resource
limits for specified method/url combinations. These limits
include such items as max concurrent connections, max file
upload size, and timeouts waiting for connection processing.
When limits are exceeded, connections are immediately
dispatched to a dedicated "error" connection processing
pool to generate a quick error response. By default all
requests share the same default limits.  Coupled with the
new "ns_pools" command, URL-based limits can provide for
sophisticated resource management.

On 7/26/07, Shedi Shedi <[EMAIL PROTECTED]> wrote:
>
> Its 4.5.0
>
>
> On 7/26/07, Nathan Folkman <[EMAIL PROTECTED]> wrote:
> >
> > What version of AOLserver are you running?
> >
> > On 7/26/07, Shedi Shedi < [EMAIL PROTECTED]> wrote:
> >
> > > Hi
> > >
> > > Patform: suse 10.1 (2.6.16.21-0.25-default)
> > >
> > > I have configured my nsd config.tcl as below:
> > >
> > > ns_param   maxconnections  100   ;# Max connections to put on
> > > queue
> > > ns_param   maxdropped  0 ;# Shut down if dropping too many
> > > conns
> > > ns_param   maxthreads  50;# Tune this to scale your server
> > > ns_param   minthreads  20 ;# Tune this to scale your
> > > server
> > > ns_param   threadtimeout   30   ;# Idle threads die at this rate
> > >
> > >
> > > ns_server threads command shows the min as zero, max as 10.
> > >
> > > Server Threads
> > > min0
> > > max10
> > > current1
> > > idle0
> > > stopping0
> > >
> > > I'm not sure what i'm missing here.
> > >
> > > regards,
> > > shedi
> > >
> > >
> > > --
> > > AOLserver - http://www.aolserver.com/
> > >
> > >
> > > To Remove yourself from this list, simply send an email to <[EMAIL 
> > > PROTECTED]> with the
> > >
> > >
> > > body of "SIGNOFF AOLSERVER" in the email message. You can leave the 
> > > Subject: field of your email blank.
> > >
> > >
> >
> >
> > --
> > Nathan Folkman
> > [EMAIL PROTECTED]
> >
> >
> > --
> > AOLserver - http://www.aolserver.com/
> >
> >
> > To Remove yourself from this list, simply send an email to <[EMAIL 
> > PROTECTED]> with the
> >
> > body of "SIGNOFF AOLSERVER" in the email message. You can leave the 
> > Subject: field of your email blank.
> >
> >
>
> --
> AOLserver - http://www.aolserver.com/
>
>
> To Remove yourself from this list, simply send an email to <[EMAIL 
> PROTECTED]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
> field of your email blank.
>
>


-- 
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command

[Nathan Folkman]

What version of AOLserver are you running?

On 7/26/07, Shedi Shedi <[EMAIL PROTECTED]> wrote:
>
> Hi
>
> Patform: suse 10.1 (2.6.16.21-0.25-default)
>
> I have configured my nsd config.tcl as below:
>
> ns_param   maxconnections  100   ;# Max connections to put on queue
> ns_param   maxdropped  0 ;# Shut down if dropping too many
> conns
> ns_param   maxthreads  50;# Tune this to scale your server
> ns_param   minthreads  20 ;# Tune this to scale your server
> ns_param   threadtimeout   30   ;# Idle threads die at this rate
>
>
> ns_server threads command shows the min as zero, max as 10.
>
> Server Threads
> min0
> max10
> current1
> idle0
> stopping0
>
> I'm not sure what i'm missing here.
>
> regards,
> shedi
>
>
> --
> AOLserver - http://www.aolserver.com/
>
>
> To Remove yourself from this list, simply send an email to <[EMAIL 
> PROTECTED]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
> field of your email blank.
>
>


-- 
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] Example of nsproxy usage

[Nathan Folkman]

From the AOLserver control port:


server1:nscp 1> set handle [ns_proxy get myProxy]
myProxy-proxy-0

server1:nscp 2> ns_proxy eval $handle [list exec uname -a]
Darwin MacBook-Pro.local 8.10.1 Darwin Kernel Version 8.10.1: Wed May 23
16:33:00 PDT 2007; root:xnu-792.22.5~1/RELEASE_I386 i386 i386

Enjoy!

- n

On 7/13/07, JAMSHED QURESHI <[EMAIL PROTECTED]> wrote:


 Helo,
Anyone like to give a taste of nsproxy usage for example an executable.

cheers,


--
AOLserver - http://www.aolserver.com/


To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.





--
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] Tcl exec hangs

[Nathan Folkman]

You might also want to take a look at using an nsproxy. The nsproxy is
essentially just a single-threaded tclsh that you can send commands to from
your AOLserver. So what you could do is let the nsproxy perform the actually
exec calls for you. Since the proxy is single-threaded, you shouldn't see
the issue you are seeing when trying to do it from within AOLserver. Let me
know if you need some example code. Hope that helps!

- n

On 7/10/07, Jeff Hobbs <[EMAIL PROTECTED]> wrote:


Titi Alailima wrote:
> Periodically some of our AOLserver installations get into a mode where
> all calls to "exec" just hang, not really taking up any processor time
> but eating up a thread.  Doesn't seem to be any memory problems
> coinciding, which I had originally suspected being a limiting factor on
> the forks.  Besides, we usually get an error message when there's not
> enough memory to fork a child process.  Haven't yet figured out anything
> in particular that causes this to start happening, but once it starts
> happening, it keeps going eating up more and more threads until we
> restart the nsd.  Any ideas what the cause might be and what I can do to
> investigate further?
>
> We run AOLserver 4.0.10 on RHELAS 3, and CentOS 4, w/ Tcl 8.4.11.

This is a known issue/conflict with pthreads and fork.  Tcl does not yet
have the pthread_at_fork functionality in place because that requires a
careful marshaling of mutexes over the fork.  You will find several core
bug reports on this in the Tcl SF db area.  It is possible to solve, its
just that noone has invested the time yet.

Jeff


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <
[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the
Subject: field of your email blank.





--
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] Aolserver 4.0.10 and Hoard 2.1.2d

[Nathan Folkman]

You'll probably also want to make sure you're building Tcl such that it
isn't using the "Zippy" threaded allocator code. Not sure if the Tcl folks
ever fixed this, but it used to be that there was no easy way to build
threaded Tcl without getting the "Zippy" allocator built as well. Using the
Tcl threaded allocator may mask any benefits you'd see from using something
like Hoard of TCMalloc since it maintains it's own per-thread memory pools
and what not.

- n

On 5/24/07, Dossy Shiobara <[EMAIL PROTECTED]> wrote:


On 2007.05.24, Agnieszka Kuka?owicz <[EMAIL PROTECTED]> wrote:
> Ns_TlsGet: invalid key: 0: should be between 1 and 100

This used to be a problem in older AOLserver 4.0, where the IP address
you're using was missing reverse DNS ... but yours is failing in
NsInitConf() ... are you sure you're loading the correct libnsd.so?  I'd
check your LD_LIBRARY_PATH and be extra-careful.

-- Dossy

--
Dossy Shiobara  | [EMAIL PROTECTED] | http://dossy.org/
Panoptic Computer Network   | http://panoptic.com/
  "He realized the fastest way to change is to laugh at your own
folly -- then you can let go and quickly move on." (p. 70)


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <
[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the
Subject: field of your email blank.





--
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] Loading and initializing Tcl packages at startup

[Nathan Folkman]

Along with this, I'd like to figure out what "ns_ictl once" actually does...
;-) I see that it adds items to a hash, but I never see any code acting on
those hashed items. Perhaps this is the foundation for the "init once"
functionality being sought? If so, we should complete the work in the head.

- n

On 5/7/07, Michael Andrews <[EMAIL PROTECTED]> wrote:


There are times when you need to load Tcl packages @ startup.  There
are also times when you might want to have a one-time
"initialization" of those packages before the server starts accepting
connections. For example, you might have a package that registers a
filter or a proc.

The current implementation "ns_ictl package require" will re-require
packages and re-eval namespaces on every interp clone. This is good
for some reasons and should not be changed, but it does not give us a
"one-time" initialization.

I have added a tcl file to my AOLserver installation called
"packages.tcl" that does the following:

1) Reads the nsd config to set library paths and load needed packages.
2) Looks for and executes the special command ::$
{packageName}::nsInit after ALL packages have been loaded.
3) Looks for and executes the special command ::$
{packageName}::nsPostInit after ALL nsInit procs have been executed.

The config look like this:

ns_section "ns/server/server1/packages"
 ns_param librarylist [list /Users/Michael/builds/aolserver/
packages]
 ns_param packagelist [list rpc TclCurl foo bar]

This has really helped me pre-load packages for a core distribution
and allowed me to use package deployments for AOLserver level code
like "ns_register_proc" and "ns_pool set" becase I could start pools
and register them before the server accepted connections.

I want to get a sense of how folks need to load packages using
AOLserver in hopes of adding this functionality to the AOLserver
source. So I have some questions:

1) Is there a need to add package loading (like what I have done) to
the core AOLserver distribution?
2) Is there a need to have a nsInit proc per package and when should
that proc be executed:
 a) Right after right the package has been loaded - and before
the other packages are loaded? (basically in line with the load list).
 b) After all the packages have been loaded (as I am doing in above)
3) Is there a need for a nsPostInit proc and when should it be executed:
 a) After all the nsInit procs have been executed but before the
server accepts connections? (as I am doing above).
 b) After all the nsInit procs have been executed but AFTER the
server accepts connections?
4) Should package load order be respected?
5) Are there other ns* procs that would be useful?
6) Other ideas on the config structure?

I personally do not code in a way that needs load order, but others
might, so my own opinion is that: Load order is easy to keep so why
not; nsInit procs should be fired after ALL packages are loaded; and
nsPostInit procs should be fired after ALL nsInits have been
executed.  This would give a nice way to initialize packages before
the server accepts connections.

I also think that any triggers that need to happen like
"ns_atshutdown" can be registered in one of the init procs so we do
not need to complicate things by have many "special" commands.

Thoughts?

M


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <
[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the
Subject: field of your email blank.





--
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] ns_db is not working

[Nathan Folkman]

Make sure you are loading the nsdb.so module.

Example:

ns_section "ns/server/server1/modules"
   ns_param nsdb nsdb.so

Hope that helps!

- n

On 5/3/07, Thorpe Mayes <[EMAIL PROTECTED]> wrote:


I have recently installed aolserver 4.5.

Everything seems to be working, except I get this error when I try to use
the ns_db command:

error: Tcl exception:
invalid command name "ns_db"
while executing
"ns_db gethandle $pool"
invoked from within
"set db [ns_db gethandle $pool]"

ns_info works fine.

I know I am missing the obvious.

Thorpe


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <
[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the
Subject: field of your email blank.





--
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] ns_zlib compress

[Nathan Folkman]

I'd have to look again to confirm whether or not ns_return wasn't binary
safe, but I wouldn't be surprised. I recently patched ns_http to correctly
handle the sending and receiving of binary data. I'll try and take a look at
as ns_return next. Would you mind adding this as a bug? Thanks!

- n

On 4/19/07, Rusty Brooks <[EMAIL PROTECTED]> wrote:


I don't believe you can use ns_return with binary data.

By the way, are there any plans to change that?  Or make a
ns_return_binary or something?  I have a lot of code that writes to a
file and then does ns_returnfile

Rusty

John Buckman wrote:
> How do you enable gzip page compression on ns_register_proc pages?
>
> The adp gzip compression occurs in C code, in Ns_ConnFlush(), which
> checks for "Accept-Encoding: gzip"
>
> Since a "ns_register_proc" uses "ns_return", like this:
> ns_return $conn 200 text/html $html
>
> I believe it's bypassing this gzip code, and always returning html.
>
> I wrote a ns_return_gzipped() proc (see below), but it doesn't seem to
> work, so I was wondering if anyone's done this already, and how.
>
> FYI I switched BookMooch to use adp gzip compression, and the results
> are impressive.  200k html files go to 35k, greatly speeding up the
> user experience.
>
> -john
>
>
>
> proc ns_return_gzipped {conn html} {
>
> if {[string first {gzip} [ns_set get [ns_conn headers]
> {Accept-Encoding}]] != -1} {
> set zl [ns_zlib gzip $html]
> ns_set put [ns_conn outputheaders] Content-Encoding gzip
> ns_return $conn 200 text/html $zl
> } else {
> ns_return $conn 200 text/html $html
> }
>
> }
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to
> <[EMAIL PROTECTED]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> Subject: field of your email blank.


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <
[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the
Subject: field of your email blank.





--
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] (Resolved) Re: nsthreadtest and segmentation fault

[Nathan Folkman]

That's just a hunch about that being the cause, but it is true, AOLserver
has not been thoroughly reviewed or tested as to be truly 64-bit clean, so
you may see issues.

- n

On 4/17/07, aT <[EMAIL PROTECTED]> wrote:


I found out (thanks to Nathan) that nsthreadtest is crashing on my servers
because i compiled aolserver with --enable-64bit .
aolserver 64 bit code is still not thoroughly tested no wonder its
crashing on my 64 bit server . On the same machine running 32bit compiled
versions works fine .
so I will avoid compiling aolserver with --enable-64bit for now.
However i must mention this here that i have run benchmarks with and
without 64bit on aolserver and found 15% to 20% performance enhancement . i
might have to do that again  :)

Thanks





On 4/15/07, aT <[EMAIL PROTECTED]> wrote:
>
> Running  nsthreadtest always produces segmentation fault on my servers.
> I have tried with Aolserver 4.0.10 and aolserver 4.5.0 on , amd x86_64
> running 2.6.9-42.ELsmp  AND  2.4.21-27.ELsmp
>
> doing gdb on core file gives this
>
> aolserver was compiled with --enable-threads
>
>
> Loaded symbols for /lib64/ld-linux-x86-64.so.2Core was generated by
> `./bin/nsthreadtest'.
> Program terminated with signal 11, Segmentation fault.
> #0 0x002a95a58bd8 in pthread_join () from /lib64/tls/libpthread.so.0
>
>
>
> --
> Syed Atif Ali
> Desk: 971 4 3911914




--
Syed Atif Ali
Desk: 971 4 3911914


--
AOLserver - http://www.aolserver.com/


To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.





--
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] profiling an aolserver site

[Nathan Folkman]

It *should* handle all connection requests.

The timing happens in the LogTrace() funtion in nslog/nslog.c. That trace
generally fires last to do logging to the access log, and the request
execution time is computed by comparing the time that LogTrace() was called
to the time returned by Ns_ConnStartTime().

Probably is a better way to do this, but this will at least give you a
consistent baseline to compare page execution times.

I don't have any analysis scripts, but would suggest looking at R.

- n

On 4/11/07, John Buckman <[EMAIL PROTECTED]> wrote:


I added server-side logging of ADP page execution a while back. Maybe that
would help you?

ns_section "ns/server/server1/module/nslog"
ns_param logreqtime true


Yes, that's great.

Will it also log ns_register_proc pages?

Have you written anything to crunch the numbers? If not, no problem, I can
do that.

-john



[MacBook-Pro:~] nfolkman% tail -f
/usr/local/aolserver/servers/server1/modules/nslog/access.log
192.168.1.129 - - [11/Apr/2007:10:46:28 -0400] "GET /flash/ HTTP/1.1" 200
191 "" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv: 1.8.1.3)
Gecko/20070309 Firefox/2.0.0.3" 0.106342
192.168.1.129 - - [11/Apr/2007:10:46:28 -0400] "GET /favicon.ico HTTP/1.1"
404 534 "" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv: 1.8.1.3)
Gecko/20070309 Firefox/2.0.0.3" 0.000361

On 4/11/07, John Buckman < [EMAIL PROTECTED]> wrote:
>
> I'm wondering if anyone on this list has written code to "profile" a
> web site running under Aolserver.
>
> By this, I mean, timing the start/stop time of every page, logging
> it, and then running a bit of analysis to find out what pages are the
> slowest running and which pages are the most commonly loaded, then
> multiplying the two (ie execution time x requests per day=total
> machine load per page per day)
>
> In traditional systems programming, profiling is a common tool used
> to determine what code should be optimized.   I'd like to do the same
> inside aolserver.
>
> One efficient alternative I was thinking about would be to patch
> ns_log to include both the start request time, and the time the page
> was returned, in the log.  That could be done if ns_log is called
> after the page is rendered, and I don't know if that's the case.
>
> -john
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to <
> [EMAIL PROTECTED]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> Subject: field of your email blank.
>



--
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/


To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.



--
AOLserver - http://www.aolserver.com/


To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.





--
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] profiling an aolserver site

[Nathan Folkman]

I added server-side logging of ADP page execution a while back. Maybe that
would help you?

ns_section "ns/server/server1/module/nslog"
   ns_param logreqtime true

[MacBook-Pro:~] nfolkman% tail -f
/usr/local/aolserver/servers/server1/modules/nslog/access.log
192.168.1.129 - - [11/Apr/2007:10:46:28 -0400] "GET /flash/ HTTP/1.1" 200
191 "" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.3)
Gecko/20070309 Firefox/2.0.0.3" 0.106342
192.168.1.129 - - [11/Apr/2007:10:46:28 -0400] "GET /favicon.ico HTTP/1.1"
404 534 "" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.3)
Gecko/20070309 Firefox/2.0.0.3" 0.000361

On 4/11/07, John Buckman <[EMAIL PROTECTED]> wrote:


I'm wondering if anyone on this list has written code to "profile" a
web site running under Aolserver.

By this, I mean, timing the start/stop time of every page, logging
it, and then running a bit of analysis to find out what pages are the
slowest running and which pages are the most commonly loaded, then
multiplying the two (ie execution time x requests per day=total
machine load per page per day)

In traditional systems programming, profiling is a common tool used
to determine what code should be optimized.   I'd like to do the same
inside aolserver.

One efficient alternative I was thinking about would be to patch
ns_log to include both the start request time, and the time the page
was returned, in the log.  That could be done if ns_log is called
after the page is rendered, and I don't know if that's the case.

-john


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <
[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the
Subject: field of your email blank.





--
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] couldn't load file "libnszlib.so": /software/aol/aol45/lib/libnszlib.so: undefined symbol: compressBound

[Nathan Folkman]

Make sure the path to the dependent libraries is in your LD_LIBRARY_PATH.

- n

On 4/10/07, aT <[EMAIL PROTECTED]> wrote:


 I have done that , I tried both ways of loading nszlib , module and
library way . Even then i get the same error.

Any Hint about the Error ?




Dossy Shiobara wrote:

On 2007.04.10, aT <[EMAIL PROTECTED]> <[EMAIL PROTECTED]> wrote:

Do i need load it from configuration file where other modules are
   loaded:  ns_section ns/server/test/modules  or i can just do "load
   libnszlib.so" in the application ?

 You have to load it in the modules section in the config.

-- Dossy


--
Syed Atif Ali
D. +971 4 3911914
F. +971 4 3911915
___
The world is coming to an end!  Repent and return those library books!


--
AOLserver - http://www.aolserver.com/


To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.





--
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] AolServer and core dumps

[Nathan Folkman]

You might want to take a look at the Tcl Core Dumper package Dossy and I did
a while back.

http://aolserver.cvs.sourceforge.net/aolserver/tcl-coredumper/

- n

On 3/28/07, Agnieszka Kukałowicz <[EMAIL PROTECTED]> wrote:


Hi,

I have question on how to force Aolserver to dump core?
My Aolserver crashes in some circumstances causing segmentation fault.
I would like to debug the problem, but I can't see any core file.
It looks like everything is set right.
I made 'ulimit -c unlimited' to have unlimited core file size and my
permissions where I dump a core look ok. What might be the reason it
does not work?

Cheers
Agnieszka Kukalowicz




--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <
[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the
Subject: field of your email blank.





--
Nathan Folkman
[EMAIL PROTECTED]

Re: [AOLSERVER] /tmp/files disappearing after request is processed

[Nathan Folkman]

Take a look at:

   ns_getform

Looks like it's getting removed here:

form.tcl:

   160 ns_atclose "ns_unlink -nocomplain $tmpfile"

Can you update your code to copy the returned /tmp file to your own
directory? Probably safer that way anyways, since you never know when /tmp
will be cleaned up.

- n


On 3/1/07, Pedro Liska <[EMAIL PROTECTED]> wrote:


Hi!

I'm moving my web server from AOLserver 3.3 to AOLserver 4.5.

With my old AOLserver 3.3 configuration, the /tmp/files posted by a
multipart/form-data form would remain after the request was processed.
This is not the case in my AOLserver 4.5 test configuration.

Do you guys know how to make AOLserver 4.5 not delete the /tmp/files
after the request is processed? My Tcl code depends on the files being
there in several cases.

Thanks!

Pedro


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <
[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the
Subject: field of your email blank.





--
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] tdom leak fix

[Nathan Folkman]

Cool.

BTW, I wouldn't call this a leak, and would instead call it misuse of tDOM.
;-)

On 2/28/07, Jeff Rogers <[EMAIL PROTECTED]> wrote:


Nathan Folkman wrote:
> Don't you open yourself up for a race condition if that code is in a
> scheduled proc?
>

If it was an aolserver scheduled proc then yes, but in my application it
was running in an "after" rescheduling loop within tclhttpd so there was
no risk of pre-emption; but the memory (dom object) issue was pretty
much the same.

-J


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <
[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the
Subject: field of your email blank.





--
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] tdom leak fix

[Nathan Folkman]

I believe ns_ictl is what you would want - "ns_ictl oncleanup" I believe.

Another option is to run all request through a configured "start page" and
to put the cleanup code in that .adp page. Nice thing about that approach is
that you can make changes without having to restart the server.

Configuration:

ns_section "ns/server/server1/adp"
   ns_param startpage /start.adp

start.adp:

<%
_ns_adp_include [ns_url2file [ns_conn url]]

#
# tDOM cleanup code goes here.
#
%>



On 2/28/07, Tom Jackson <[EMAIL PROTECTED]> wrote:


Yes, wouldn't it be nice if ns_atclose worked in scheduled procs? In a
conn
you can use ns_atclose, but it has been removed from scheduled procs. Is
there no way to run a trace on a scheduled proc? I have looked at ns_ictl,
but it looks like the scheduling is per module, so you can't just activate
it
when you want like ns_atclose.

Not having to catch code is a big benefit, being able to use the same code
in
all threads would be very nice.

tom jackson

On Wednesday 28 February 2007 11:41, Nathan Folkman wrote:
> Don't you open yourself up for a race condition if that code is in a
> scheduled proc?


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <
[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the
Subject: field of your email blank.





--
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] tdom leak fix

[Nathan Folkman]

Don't you open yourself up for a race condition if that code is in a
scheduled proc?

On 2/28/07, Jeff Rogers <[EMAIL PROTECTED]> wrote:


Nick Miller wrote:
> Heya,
>
> I have added:
>
> foreach lostdom [info commands domDoc0*] {
>$lostdom delete
> }
>
> As you say though, best to make sure you catch them when your dealing
> with them and explicitly delete them.
>
> Thanks
> Nick

Having run into this same problem before I did something similar,
setting up the above as a scheduled proc.  Looking at my code I also
included

 eval unset [info vars ::dom::_node*Attributes]

although I don't recall if this was for tdom or tclxml.

-J


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <
[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the
Subject: field of your email blank.





--
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] Memory Leaks of tDOM

[Nathan Folkman]

Well, there's the "ns_info pools" Tcl command. Also look at
aolserver/modules/tcl/stats.tcl for example usage.

On 2/27/07, Nick Miller <[EMAIL PROTECTED]> wrote:

Hey Nathan,

I have run into an issue with yea oldie memory leak problem. Was
chatting to Dave about this earlier on and am trying to find something
to quickly get usable memory usage information back on AOLServer. I am
not sufficiently skilled in the ways of memory use by Tcl to be able to
quickly put something together and wondered if you knew of anything that
might be a useful tool for this? Mr Schroeder mentioned that once upon a
time we had some compiled c code that was used to debug memory leak
problems - do you know anything about this?

I hope all is well with you.
nick


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to
<[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject:
field of your email blank.




--
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] Ns_proxy

[Nathan Folkman]

One thing you can try is to have your proxies start up when the server
starts. You can warm them up by sending down a command at server
startup.

On 2/25/07, Malte Sussdorff <[EMAIL PROTECTED]> wrote:

Due to the fact that we are experiencing a couple of problems with exec in
AOLserver 4.5 with TCL 8.4.14 due to out of memory, I wanted to run the TCL
exec command with nsproxy.

Sadly, ns_proxy get "test" fails as well with "not enough memory".

Now, I am wondering, what else can be done. The memory is at 4GB at the
moment.

There might be an issue with the OpenACS site_node cache, but this is a
nsv_array, so it should not be replicated in the exec process, or am I
wrong?

Any hint is highly welcome, maybe I need to setup ns_proxy Pools just like
database pools ?

Thanks
 Malte


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to
<[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject:
field of your email blank.




--
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] mutex try lock?

[Nathan Folkman]

For John. Just curious as to what problem the "try lock" was needed for.

- n

On 2/22/07, Tom Jackson <[EMAIL PROTECTED]> wrote:


Nathan,

Was this a question for me, or John?

tom jackson

On Thursday 22 February 2007 08:18, Nathan Folkman wrote:
> Just curious, but what's the use case where you would use something like
> this?
>
> - n
>
> On 2/22/07, Tom Jackson <[EMAIL PROTECTED]> wrote:
> > I have some example code of condition wait:
> >
> >
> >
http://rmadilo.com/m2/servers/rmadilo/modules/tcl/twt/packages/db/tcl/dat
> >asource-procs.tcl
> >
> > The procs lock, unlock, wait, popPool, etc. Wrap the ideas up pretty
> > easily,
> >
> > but the question is what happens when you code times out waiting for a
> > resource?
> >
> > tom jackson
> >
> > On Thursday 22 February 2007 07:09, John Buckman wrote:
> > > I'm looking to have mutexes with timeouts, and I see support in the
C
> > > code for this but none carried over to Tcl.
> > >
> > > In the C code, there's a Ns_MutexTryLock() function, but no tcl
> > > function for calling it.
> > >
> > > Ns_MutexLock calls Ns_MutexTryLock() and there appears to be timeout
> > > support:
> > >  if (!NsLockTry(mutexPtr->lock)) {
> > >   return NS_TIMEOUT;
> > >  }
> > >
> > > but I don't see any way of setting the mutex timeout seconds.
> > >
> > > It's trivially simple to modify NsTclMutexObjCmd (in tclthread.c) to
> > > support "ns_mutex try" and I'm wondering if there's a reason this
> > > hasn't been done.
> > >
> > > -john
> > >
> > >
> > > --
> > > AOLserver - http://www.aolserver.com/
> > >
> > > To Remove yourself from this list, simply send an email to
> > > <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in
the
> > > email message. You can leave the Subject: field of your email blank.
> >
> > --
> > AOLserver - http://www.aolserver.com/
> >
> > To Remove yourself from this list, simply send an email to <
> > [EMAIL PROTECTED]> with the
> > body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> > Subject: field of your email blank.


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <
[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the
Subject: field of your email blank.





--
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] mutex try lock?

[Nathan Folkman]

Just curious, but what's the use case where you would use something like
this?

- n

On 2/22/07, Tom Jackson <[EMAIL PROTECTED]> wrote:


I have some example code of condition wait:


http://rmadilo.com/m2/servers/rmadilo/modules/tcl/twt/packages/db/tcl/datasource-procs.tcl

The procs lock, unlock, wait, popPool, etc. Wrap the ideas up pretty
easily,

but the question is what happens when you code times out waiting for a
resource?

tom jackson


On Thursday 22 February 2007 07:09, John Buckman wrote:
> I'm looking to have mutexes with timeouts, and I see support in the C
> code for this but none carried over to Tcl.
>
> In the C code, there's a Ns_MutexTryLock() function, but no tcl
> function for calling it.
>
> Ns_MutexLock calls Ns_MutexTryLock() and there appears to be timeout
> support:
>  if (!NsLockTry(mutexPtr->lock)) {
>   return NS_TIMEOUT;
>  }
>
> but I don't see any way of setting the mutex timeout seconds.
>
> It's trivially simple to modify NsTclMutexObjCmd (in tclthread.c) to
> support "ns_mutex try" and I'm wondering if there's a reason this
> hasn't been done.
>
> -john
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to
> <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the
> email message. You can leave the Subject: field of your email blank.


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <
[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the
Subject: field of your email blank.





--
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] TDOM and OSX

[Nathan Folkman]

1. Check and see if "ns_info pools" returns any data.
2. Here's one option: http://developer.apple.com/macosx/launchd.html

- n

On 2/21/07, John Divney <[EMAIL PROTECTED]> wrote:


 This did the trick.  Thanks for all the suggestions.

A couple more questions:

1) How do I know zippy is working. I did compile TCL myself.  Is there
something I can try to verify ?

2) What is equivalent of init.tab on OS X ?

Thanks!

div

Michael Andrews wrote:


 I had this same issue.  You need to get and compile tdom from CVS. Do not
get it from a mirror on http://www.tdom.org/

 Do this:

 cvs -d:pserver:[EMAIL PROTECTED]:/usr/local/pubcvs login
cvs -z3 -d:pserver:[EMAIL PROTECTED]:/usr/local/pubcvs co tdom
../configure --enable-symbols --with-tcl=/usr/local/aolserver/lib
--prefix=/usr/local/aolserver --with-aolserver=/usr/local/aolserver

 That'll fix your issue.

 On Feb 21, 2007, at 12:48 PM, John Divney wrote:

 Hi,

 I am having trouble w/ tdom on OS X.  I am able to run make/make install
w/out issue.
This copies the libraries to the aolserver directory, but when I start
AOLServer I get the following error:

 [21/Feb/2007:12:44:52][382.2684407744][-main-] Error: tcl: source
/usr/local/aolserver/modules/tcl/init.tcl failed: couldn't find procedure
Tdom_Init
NONE
couldn't find procedure Tdom_Init
   while executing
"ns_eval package require tdom"
   (file "/usr/local/aolserver/modules/tcl/init.tcl" line 64)
   invoked from within
"source $file"

 I am calling package require from aolserver/modules/tcl/init.tcl

 The libraries are in /aolserver/lib/tdom0.8.0

 When I grep for Tdom_Init is is only in the .a file, not in the .so or
.dylib file, so I think I am doing something wrong w/ my CONFIG file.

 Here is how I am calling that.

 ../configure \
  --mandir=/usr/local/share/man \
  --enable-threads \
  --prefix=/usr/local/aolserver \
  --libdir=/System/Library/Tcl \
  --with-tcl=/System/Library/Frameworks/Tcl.framework \
  --with-tclinclude=/System/Library/Frameworks/Tcl.framework/Headers

 I have tried a ton of alternatives, read through posts on openacs site,
and some others, but nothing seems to work.
Any help is appreciated !

 Thanks!

 div


 --
AOLserver - http://www.aolserver.com/

 To Remove yourself from this list, simply send an email to <
[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the
Subject: field of your email blank.



--
AOLserver - http://www.aolserver.com/


To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
<[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.


--
AOLserver - http://www.aolserver.com/


To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.





--
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] TDOM and OSX

[Nathan Folkman]

You'll probably want to also compile Tcl yourself.

Not sure the version included with OS X is compiled with support for
threads, but you'd have to check...

Tcl:

./configure --enable-threads ...

If you don't compile Tcl with support for threads, you won't get the fast
"Zippy" allocator.

- n

On 2/21/07, Michael Andrews <[EMAIL PROTECTED]> wrote:



use your own paths for tcl and aolserver in the configure command of
course...

M

On Feb 21, 2007, at 1:22 PM, Michael Andrews wrote:


I had this same issue.  You need to get and compile tdom from CVS. Do not
get it from a mirror on http://www.tdom.org/

Do this:

cvs -d:pserver:[EMAIL PROTECTED]:/usr/local/pubcvs login
cvs -z3 -d:pserver:[EMAIL PROTECTED]:/usr/local/pubcvs co tdom
../configure --enable-symbols --with-tcl=/usr/local/aolserver/lib
--prefix=/usr/local/aolserver --with-aolserver=/usr/local/aolserver

That'll fix your issue.

On Feb 21, 2007, at 12:48 PM, John Divney wrote:

Hi,

I am having trouble w/ tdom on OS X.  I am able to run make/make install
w/out issue.
This copies the libraries to the aolserver directory, but when I start
AOLServer I get the following error:

[21/Feb/2007:12:44:52][382.2684407744][-main-] Error: tcl: source
/usr/local/aolserver/modules/tcl/init.tcl failed: couldn't find procedure
Tdom_Init
NONE
couldn't find procedure Tdom_Init
   while executing
"ns_eval package require tdom"
   (file "/usr/local/aolserver/modules/tcl/init.tcl" line 64)
   invoked from within
"source $file"

I am calling package require from aolserver/modules/tcl/init.tcl

The libraries are in /aolserver/lib/tdom0.8.0

When I grep for Tdom_Init is is only in the .a file, not in the .so or
.dylib file, so I think I am doing something wrong w/ my CONFIG file.

Here is how I am calling that.

../configure \
  --mandir=/usr/local/share/man \
  --enable-threads \
  --prefix=/usr/local/aolserver \
  --libdir=/System/Library/Tcl \
  --with-tcl=/System/Library/Frameworks/Tcl.framework \
  --with-tclinclude=/System/Library/Frameworks/Tcl.framework/Headers

I have tried a ton of alternatives, read through posts on openacs site,
and some others, but nothing seems to work.
Any help is appreciated !

Thanks!

div


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <
[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the
Subject: field of your email blank.




--
AOLserver - http://www.aolserver.com/


To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.





--
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] make install problem

[Nathan Folkman]

The AOLserver 4.5 make process is slightly different then previous versions:

% cd aolserver
% ./configure --enable-symbols --prefix=/usr/local/aolserver
--with-tcl=/usr/local/aolserver/lib
% /usr/local/aolserver/bin/tclsh8.4 nsconfig.tcl -debug -install
/usr/local/aolserver
% make
% make install



On 2/6/07, Arturo Mardones <[EMAIL PROTECTED]> wrote:


Hi!

I'm installing aolserver on mandriva 2007, and i have this packages:

libtcl8.4-devel-8.4.13-1mdk
libtcl8.4-8.4.13-1mdk
tcl-8.4.13-1mdk
and daemontools

but this output have when try install

[EMAIL PROTECTED] ~]# cd /usr/local/src/aolserver-4.5.0/
[EMAIL PROTECTED] aolserver-4.5.0]# ./configure --enable-threads
checking for Tcl configuration... found /usr/lib/tclConfig.sh
checking for existence of /usr/lib/tclConfig.sh... loading
checking for build with symbols... no
checking for gcc... gcc -pipe
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc -pipe accepts -g... yes
checking for gcc -pipe option to accept ANSI C... none needed
checking how to run the C preprocessor... gcc -pipe -E
checking for egrep... grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for inttypes.h... (cached) yes
checking for timegm... yes
checking for fork1... no
checking for drand48... yes
checking for random... yes
checking for _NSGetEnviron... no
checking for poll... yes
checking dl.h usability... no
checking dl.h presence... no
checking for dl.h... no
checking need for dup high... no
checking for shared libgcc... yes
checking for getaddrinfo in -lsocket... no
checking for getnameinfo in -lsocket... no
checking for getaddrinfo... yes
checking for getnameinfo... yes
configure: creating ./config.status
config.status: creating include/ns.mak
[EMAIL PROTECTED] aolserver-4.5.0]# make install
./util/nsmakeall.tcl install nsthread nsd nstclsh
./util/nsmakeall.tcl: line 48: syntax error near unexpected token `}'
./util/nsmakeall.tcl: line 48: `}'
make: *** [install-bins] Error 2

Thanks for any help!!



--

http://animaldelared.blogspot.com


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <
[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the
Subject: field of your email blank.





--
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] ns_adp_ctl

[Nathan Folkman]

We're behind on the 4.5 documentation... Look for things to pick up
with 4.5 development soon. ;-)

On 1/12/07, Gustaf Neumann <[EMAIL PROTECTED]> wrote:

Dear all.

Is it intentional that  ns_adp_ctl is not
mentioned on http://panoptic.com/wiki/aolserver/Tcl_API ?

One gets no friendly feedback,
 "no adp output context"
when one tries to ns_adp_parse a file larger than 1024000 bytes.
It took me a while to find the reason.

the 4.5 function "ns_adp_ctl bufsize" can be easily
increased for such cases (default 1024000).

other options of ns:adp_ctl are:
"bufsize", "channel",
"autoabort", "detailerror", "displayerror", "expire", "gzip",
"nocache", "safe", "singlescript", "stricterror", "trace",
"trimspace",


best regards

 -gustaf neumann


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to
<[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject:
field of your email blank.




--
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] AOLserver 4.0.10 and Hoard

[Nathan Folkman]

Are you trying to preload Hoard with a version of Tcl that was built with
the threaded allocator? If so, you're not going to realize much in the way
of performance or memory savings since higher-level allocations will still
ultimately be going through Tcl's threaded (bucket) allocator.

On 1/2/07, Agnieszka Kukałowicz <[EMAIL PROTECTED]> wrote:


Hi,

I have problem with running AOLserver 4.0.10 with Hoard on my production
server.
After putting in my run script the line below:
export
LD_PRELOAD="/usr/lib/libdl.so:/usr/local/lib/libhoard.so:/usr/lib/libpth
rea.so"

and restarting the server I get:

Ns_TlsGet: invalid key: 0: should be between 1 and 100

When I tested Hoard on my developer server everything was ok.

What am I doing wrong?

Cheers

Agnieszka Kukałowicz




--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <
[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the
Subject: field of your email blank.





--
Nathan Folkman
[EMAIL PROTECTED]

Re: [AOLSERVER] aolserver bug

[Nathan Folkman]

Saw. Cool. Thanks!

On 12/15/06, Dossy Shiobara <[EMAIL PROTECTED]> wrote:


On 2006.12.14, Gustaf Neumann <[EMAIL PROTECTED]> wrote:
> I am following up a posting from January, describing a problem
> which lead to frequent hangs on openacs.org.
[...]
> i would recommend to add this or a similar fix to the
> code base of aolserver 4.5, since the reported problem
> persists as well in the new driver of 4.5

I've gone and accepted the patch for both HEAD and aolserver_v40_bp.
Refer to SF BUG #1615787:

http://aolserver.com/sf/bug/1615787

Thanks!

-- Dossy

--
Dossy Shiobara  | [EMAIL PROTECTED] | http://dossy.org/
Panoptic Computer Network   | http://panoptic.com/
  "He realized the fastest way to change is to laugh at your own
folly -- then you can let go and quickly move on." (p. 70)


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <
[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the
Subject: field of your email blank.





--
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] tcmalloc

[Nathan Folkman]

There actually is a fair amount of documentation, just most of it is in the
source file. Take a look at:

zippy2/README
zippy2/tclThreadAlloc.c

Zippy2 allows you to override the Tcl threaded allocator functions, so you
don't need to worry about recompiling Tcl with "USE_THREAD_ALLOC=0" in this
case.

To run, you'd do something like:

env -i LD_PRELOAD=libhoard.so:libzippy.so TCL_ALLOC_DISABLE=1 bin/nsd -ft
nsd.tcl

The Zippy2 allocator is a complete work in progress, and has never been
deployed in production. It might be better to look at the performance of
threaded Tcl without the thread allocator (USE_THREAD_ALLOC=0) and hoard.

Hope that helps!

On 12/14/06, Agnieszka Kukałowicz <[EMAIL PROTECTED]> wrote:


 For sure…

Followed the instructions for Hoard for example, I ony set the LD_PRELOAD
variable.

And there is also no instructions for zippy2 to recompile TCL without
DUSE_THREAD_ALLOC.

So my question is if I want to use Hoard/Zippy2/Tcmalloc do I need to
built TCL without the zippy memory allocator as you wrote?



And the second thing…

Maybe, It would be a good idea to put these instruction in README file in
Zippy2 code?





Agnieszka





-Original Message-
*From:* AOLserver Discussion [mailto:[EMAIL PROTECTED] *On
Behalf Of *Nathan Folkman
*Sent:* Wednesday, December 13, 2006 8:07 PM
*To:* AOLSERVER@LISTSERV.AOL.COM
*Subject:* Re: [AOLSERVER] tcmalloc



Exactly right. Thanks for jogging my memory! ;-) Here's a snippet from one
of our custom build scripts which might help:

@echo "** MODIFYING TCL TO BUILD WITHOUT THE ZIPPY MEMORY ALLOCATOR
**"
cd $(ROOT_DIR)/tcl/unix && \
cat Makefile | sed -e 's/-DUSE_THREAD_ALLOC=1//' > Makefile.new && \
cat tclConfig.sh | sed -e 's/-DUSE_THREAD_ALLOC=1//' >
tclConfig.sh.new && \
cp Makefile.new Makefile && \
    cp tclConfig.sh.new tclConfig.sh

On 12/13/06, *Jeff Rogers* <[EMAIL PROTECTED]> wrote:

Nathan Folkman wrote:

> What you need to do is build a version of Tcl that has support for
threads,
> but does not use the "Zippy" allocator. I seem to remember there being
> issues with core Tcl that did not allow you to do this easily, but you'd

> have to double check. Maybe try something like "--enable-threads
> --disable-thread-alloc."

If I recall from when I was poking at this, there is no configure flag
to disable the threaded allocator; instead you need to edit the
generated Makefile after you run "configure" and remove
-DUSE_THREAD_ALLOC=1 from AC_FLAGS.  And of course, 'make clean' before
remaking or nothing will change.

-J


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <
[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the
Subject: field of your email blank.




--
Nathan Folkman
[EMAIL PROTECTED]





--

AOLserver - http://www.aolserver.com/



To Remove yourself from this list, simply send an email to <
[EMAIL PROTECTED]> with the

body of "SIGNOFF AOLSERVER" in the email message. You can leave the
Subject: field of your email blank.


--
AOLserver - http://www.aolserver.com/


To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.





--
Nathan Folkman
[EMAIL PROTECTED]

Re: [AOLSERVER] aolserver bug

[Nathan Folkman]

I'll take a look at this over the next couple of weeks when I'm on vacation
and see if I can get it applied. Thanks for finding and fixing! :-)

On 12/14/06, dhogaza@pacifier.com  wrote:


> i would recommend to add this or a similar fix to the
> code base of aolserver 4.5, since the reported problem
> persists as well in the new driver of 4.5

Let me second Gustaf on this.  Our site (openacs.org) was locking up every
few hours for some weeks before we applied this patch.  After applying it,
we've gone two full days without lock-up.

This bug is real, and this or another fix really need to get into the
code.


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <
[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the
Subject: field of your email blank.





--
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

[AOLSERVER] AOLserver Memory Allocator Performance

[Nathan Folkman]

Couldn't resist looking at this... :-)

Linux localhost.localdomain 2.6.9-34.EL #1 Fri Feb 24 16:44:51 EST 2006 i686
i686 i386 GNU/Linux

AOLserver 4.5 + Tcl 8.4.14 (USE_THREAD_ALLOC=0)

% bin/nsthreadtest

starting 10 malloc threads...waitingdone:  0 seconds, 159637 usec
starting 10 ns_malloc threads...waitingdone:  0 seconds, 102511 usec


AOLserver 4.5 + Tcl 8.4.14 (USE_THREAD_ALLOC=1)

% bin/nsthreadtest

starting 10 malloc threads...waitingdone:  0 seconds, 104061 usec
starting 10 ns_malloc threads...waitingdone:  0 seconds, 109929 usec


AOLserver 4.5 + Tcl 8.4.14 (USE_THREAD_ALLOC=0) + Google Perf Tools
(tcmalloc) 0.8.1

% env -i LD_PRELOAD=lib/libtcmalloc.so bin/nsthreadtest

starting 10 malloc threads...waitingdone:  0 seconds, 91301 usec
starting 10 ns_malloc threads...waitingdone:  0 seconds, 107320 usec

Clearly we need a better set of tests... Anyone got any ideas? Seems like
the two things we'd want to compare are performance and overall memory
usage.

--
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] tcmalloc

[Nathan Folkman]

Exactly right. Thanks for jogging my memory! ;-) Here's a snippet from one
of our custom build scripts which might help:

   @echo "** MODIFYING TCL TO BUILD WITHOUT THE ZIPPY MEMORY ALLOCATOR **"
   cd $(ROOT_DIR)/tcl/unix && \
   cat Makefile | sed -e 's/-DUSE_THREAD_ALLOC=1//' > Makefile.new && \
   cat tclConfig.sh | sed -e 's/-DUSE_THREAD_ALLOC=1//' > tclConfig.sh.new&& \
   cp Makefile.new Makefile && \
   cp tclConfig.sh.new tclConfig.sh

On 12/13/06, Jeff Rogers <[EMAIL PROTECTED]> wrote:


Nathan Folkman wrote:

> What you need to do is build a version of Tcl that has support for
threads,
> but does not use the "Zippy" allocator. I seem to remember there being
> issues with core Tcl that did not allow you to do this easily, but you'd
> have to double check. Maybe try something like "--enable-threads
> --disable-thread-alloc."

If I recall from when I was poking at this, there is no configure flag
to disable the threaded allocator; instead you need to edit the
generated Makefile after you run "configure" and remove
-DUSE_THREAD_ALLOC=1 from AC_FLAGS.  And of course, 'make clean' before
remaking or nothing will change.

-J


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <
[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the
Subject: field of your email blank.





--
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] tcmalloc

[Nathan Folkman]

Always know your audience... ;-)

Let me start over as to make sure I'm not leading anyone astray...

Tcl can be built in one of two ways: single-threaded, which is the default,
or threaded, which requires you to configure things with the
"-enable-threads" flag. When you build Tcl with support for threads, the
threaded allocator, aka "Zippy," is enabled.

At a high level, the "Zippy" allocator is a x2 allocator which is optimized
around lock avoidance that comes at the cost of relatively high memory
overhead. Ultimately the "Zippy" allocator uses malloc() and free() for
block allocations, and mmap() for large allocations, but the allocated
blocks flow back and forth between a shared pool and per-thread pools. There
is also no automatic garbage collection. The allocator basically caches
memory in both shared and per-thread pools as to avoid having to go to the
system to get memory.

If you were to try and pre-load an alternate memory allocator for threaded
Tcl whichused the "Zippy" allocator by default, you wouldn't see much
difference, since the higher memory usage is a side affect of how the
"Zippy" allocator is functions - x2 bucket allocations, and how that
allocated memory is passed between the shared and per-thread pools.

What you need to do is build a version of Tcl that has support for threads,
but does not use the "Zippy" allocator. I seem to remember there being
issues with core Tcl that did not allow you to do this easily, but you'd
have to double check. Maybe try something like "--enable-threads
--disable-thread-alloc."

The newer zippy2 code exposes an environment variable that allows you to
turn off the threaded allocator at load time.

Example:

env -i TCL_ALLOC_DISABLE=1 LD_PRELOAD=libhoard.so bin/nsd ...

Setting the "TCL_ALLOC_DISABLE" environment variable will send all
allocations directly to malloc() and free() which can then be overridden by
the "LD_PRELOAD" of an alternate memory allocator.

Hope that helps?

On 12/13/06, aT <[EMAIL PROTECTED]> wrote:


 I understand developers might be able to understand what you mentioned ,
being a sysadmin it is a bit hard for me to understand LD_PRELOAD and
compiling tcl not to use threaded allocator .

 i would really appreciate  if the above mentioned "basically" paragraph
be explained more .
 *
"Basically what you need to do is compile Tcl so that it doesn't use the
threaded allocator, and instead uses regular malloc() and free(). "*

I tried in tcl source "./configure --help" to find out how can i do what u
mentioned , but could not find any help how to , not use,  threaded
allocator and instead use malloc() and free() .


*" Then compile tcmalloc,  "*

If i compile tcmalloc is this a standalone procedure or do i need to
mention aolserver location in anyway .


*
" and finally LD_PRELOAD tcmalloc at AOLserver startup time. This will
override malloc() and free()." *

How do i do that ?


Nathan Folkman wrote:

Basically what you need to do is compile Tcl so that it doesn't use the
threaded allocator, and instead uses regular malloc() and free(). Then
compile tcmalloc, and finally LD_PRELOAD tcmalloc at AOLserver startup time.
This will override malloc() and free(). Hope that helps!

- n

On 12/11/06, aT <[EMAIL PROTECTED]> wrote:
>
> Nathan Folkman wrote:
>
> Anyone tried Google's tcmalloc with AOLserver instead of using Tcl's
> threaded allocator? Looks like folks are seeing some good things with it.
>
> http://dammit.lt/2006/12/06/google-perftools-tcmalloc-squid/
>
> --
> Nathan Folkman
> [EMAIL PROTECTED]
>
>
> --
> AOLserver - http://www.aolserver.com/
>
>
> To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
<[EMAIL PROTECTED]> with the
>
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.
>
>  I would like to try it , but have no clue how to compile this against
> aolserver.
> can anyone post a small howto on compiling it with aolserver ?
>
>
> --
> Syed Atif Ali
> D. +971 4 3911914
> F. +971 4 3911915
> ___
> The sum of the intelligence on the planet is a constant; the population is 
growing.
>
>
> --
> AOLserver - http://www.aolserver.com/
>
>
> To Remove yourself from this list, simply send an email to <[EMAIL 
PROTECTED]> with the
>
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.
>
>


--
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/


To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
<[EMAIL PROTECTED]> with the
body of &q

Re: [AOLSERVER] zippy2 Build Problem for AOLserver 4.0.x

[Nathan Folkman]

Glad it worked. :-)

The Tcl threaded allocator, which is derived from the original AOLserver
"zippy" allocator, is optimized solely for lock avoidance, and this comes at
the cost of memory foot print. In fact, there's no garbage collection for
smaller allocations. Instead, memory moves between the shared and per-thread
pools.

There are a couple of additional things you could try which may also help.
Make sure you configure your connection threads such that they eventually
exit. I believe by default they stick around forever and get re-used. This
can be a good thing, but depending on the max number of connection threads
you have configured, you may tying up a bunch of memory in the per-thread
pool. When connection threads exit, the memory from the per-thread pool is
returned to the shared pool.

Another technique which has worked well is to move memory intensive tasks
into the nsproxy. The nsproxy is a single-threaded tclsh which uses system
malloc() and free(), and which will also return memory to the system on
exit.

We played around with various options in the new "zippy2" allocator, but
never saw any major improvements, at least for the types of systems we ran
at AOL.

If you have time, you may want to try out Google's tcmalloc, as its being
getting a lot of positive feedback, and is supposed to be optimized for
threaded applications.

On 12/11/06, Agnieszka Kukałowicz <[EMAIL PROTECTED]> wrote:


Hi,

> Try the following as a replacement for the current Makefile:
>
> CLEAN = clean-it
> INSTALL = install-it
> LIB = zippy2
> LIBOBJS = tclThreadAlloc.o
> CFLAGS += -I$(TCL)/generic
> ...

It works. Thanks a lot.

> Also curious as to what motivated you to look into this in the first
> place. Are you experiencing memory issues? Are you experiencing memory

> issues?

Yes, My production server uses a lot of memory (almost 2GB after one day
running). First, I thought it was because of some kind of bad code in
our
application. But I made a lot of tests and statistics and I did notice
nothing. The application is very stable, there is no leak of memory. My
conclusion was it was due to memory alloction in AOLserver and articles
in
the Internet confirmed that. Now I'm testing AOLserver with Hoard and I
need to say it looks very nice.  The memory usage decreased about 20% -
40% depending on server parameters.
I would also like to see how zippy2 works and compare it with hoard.
I will let you know about results of my experiment.

Cheers,
Agnieszka


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <
[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the
Subject: field of your email blank.





--
Nathan Folkman
[EMAIL PROTECTED]

Re: [AOLSERVER] tcmalloc

[Nathan Folkman]

Basically what you need to do is compile Tcl so that it doesn't use the
threaded allocator, and instead uses regular malloc() and free(). Then
compile tcmalloc, and finally LD_PRELOAD tcmalloc at AOLserver startup time.
This will override malloc() and free(). Hope that helps!

- n

On 12/11/06, aT <[EMAIL PROTECTED]> wrote:


 Nathan Folkman wrote:

Anyone tried Google's tcmalloc with AOLserver instead of using Tcl's
threaded allocator? Looks like folks are seeing some good things with it.

http://dammit.lt/2006/12/06/google-perftools-tcmalloc-squid/

--
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/


To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
<[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

 I would like to try it , but have no clue how to compile this against
aolserver.
can anyone post a small howto on compiling it with aolserver ?


--
Syed Atif Ali
D. +971 4 3911914
F. +971 4 3911915
___
The sum of the intelligence on the planet is a constant; the population is 
growing.


--
AOLserver - http://www.aolserver.com/


To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.





--
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

[AOLSERVER] test

[Nathan Folkman]

test


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

[AOLSERVER] tcmalloc

[Nathan Folkman]

Anyone tried Google's tcmalloc with AOLserver instead of using Tcl's
threaded allocator? Looks like folks are seeing some good things with it.

http://dammit.lt/2006/12/06/google-perftools-tcmalloc-squid/

--
Nathan Folkman
[EMAIL PROTECTED]


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

[AOLSERVER] zippy2 Build Problem for AOLserver 4.0.x

[Nathan Folkman]

Try the following as a replacement for the current Makefile:

CLEAN   = clean-it
INSTALL = install-it
LIB = zippy2
LIBOBJS = tclThreadAlloc.o
CFLAGS  += -I$(TCL)/generic

include $(AOLSERVER)/include/Makefile.module

$(LIB): $(OBJS)
   $(LDLIB) $(LDFLAGS) -o $(LIB) $(LIBOBJS)

$(CLEAN):
   $(RM) $(LIBOBJS) $(LIB)

$(INSTALL): $(BUILD)
   $(INST) -d $(INSTLIB) -e $(LIB)

You'll need to set "TCL" to point to the Tcl source directory.

One other option, and something you'd have to do if on a Mac, is to simply
replace the tclThreadAlloc.c file in your Tcl source tree with the one
provided here, rebuild Tcl, then build the alloccmd stuff (make -f
Makefile.cmd).

Also curious as to what motivated you to look into this in the first place.
Are you experiencing memory issues?

Hope that helps!

- n


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] zippy2 problem

[Nathan Folkman]

[EMAIL PROTECTED] wrote:

Hi,

  

If memory serves, the zippy2 work was done against AOLserver 4.5,


which
  

uses slightly different semantics for building. I think you should be
able to simply change that line to be:

Makefile:41: [.]/aolserver/include/Makefile.module




Unfortunately it doesn't work. While building, I get errors:

tclThreadAlloc.c:19:20: tclInt.h: No such file or directory
tclThreadAlloc.c:20:21: tclPort.h: No such file or directory

I guess it is because of a problem to find TCL include directory.
But I don't know what to change in order to make it work. 


Please, help.

Cheers 
Agnieszka Kukalowicz



--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.
  

Are you trying to build this for AOLserver 4.0.x?

- n


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] zippy2 problem

[Nathan Folkman]

[EMAIL PROTECTED] wrote:


Hi,

I have problem compiling zippy2 allocator.

I hoped to make some tests to compare Zippy2 with Hoard allocator,

but when I tried to build zippy2 I get errors:

Makefile:41: […]/aolserver/include/ns.mak: No such file or directory

I think it is due to missing ns.mak file in version 4.0.10 I have.

So, my question is what I should change in Makefile for to build 
zippy2 without errors.


I use AOLserver in 4.0.10 in my production environment and so far I 
don’t want to upgrade it


to 4.5 version.

Agnieszka Kukalowicz

-- AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to 
<[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in 
the email message. You can leave the Subject: field of your email blank.


If memory serves, the zippy2 work was done against AOLserver 4.5, which 
uses slightly different semantics for building. I think you should be 
able to simply change that line to be:


Makefile:41: […]/aolserver/include/Makefile.module

Below are a few other items which are different between 4.5 and previous 
versions. Hope that helps. Let me know if that doesn't work. Also let me 
know if you need any help getting things set up to test with hoard.


- n


   Makefile Incompatibilities

Previous server versions:

MOD = .so

LIB = 
LIBINIT = 
LIBLIBS = 
LIBOBJS = 

INSTALL = install-hdr

include $(AOLSERVER)/include/Makefile.module

AOLserver 4.5:

MOD = 

DLL = 
DLLINIT = 
DLLLIBS = 
OBJS = 

INSTALL = install-dll

include $(AOLSERVER)/include/ns.mak


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] urldecoding unicode?

[Nathan Folkman]

[EMAIL PROTECTED] wrote:

What I meant was I can fix it on the Tcl side where I'm getting the data, but 
to make it universal and transparent to Tcl coders such as myself it would have 
to fixed in the C code of things like ns_urldecode and ns_getform.  And I have 
neither the time nor the chops to take that on right now.  So I am asking if 
anyone had that in mind to do it.  If not, it could eventually prove to be a 
high enough priority for us as we move to more AJAX input, so we would revisit 
allocating some resources to it at that point.

-T.

-Original Message-
From: Nathan Folkman [mailto:[EMAIL PROTECTED] 
Sent: Wednesday, December 06, 2006 9:49 AM

To: AOLSERVER@LISTSERV.AOL.COM
Subject: Re: [AOLSERVER] urldecoding unicode?


[EMAIL PROTECTED] wrote:
  

I have some data urlencoded by JavaScript's escape function which is
turning a Unicode apostrophe into %u2019.  AOLserver doesn't recognize 
this.  I can certainly fix this in specific locations, but to handle 
this universally it seems it this would need to be be fixed in 
ns_urldecode and/or ns_getform.  Any plans to do such a thing?
 
-Titi


-- AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to
<[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in 
the email message. You can leave the Subject: field of your email blank.




Would you mind submitting a patch for this? Thanks!

- n


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the 
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of 
your email blank.


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.
  
Understand. Would you mind filing it as a bug in SourceForge so we can 
track it? Thanks!


- n


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] urldecoding unicode?

[Nathan Folkman]

[EMAIL PROTECTED] wrote:
I have some data urlencoded by JavaScript's escape function which is 
turning a Unicode apostrophe into %u2019.  AOLserver doesn't recognize 
this.  I can certainly fix this in specific locations, but to handle 
this universally it seems it this would need to be be fixed in 
ns_urldecode and/or ns_getform.  Any plans to do such a thing?
 
-Titi


-- AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to 
<[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in 
the email message. You can leave the Subject: field of your email blank.



Would you mind submitting a patch for this? Thanks!

- n


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] does aolserver run on IBM server

[Nathan Folkman]

[EMAIL PROTECTED] wrote:


Is there anybody on this mailing list

If I get in troubles, I’m afraid the other zones on the mainframe 
crashes as well as mine J


I’ll give a shot anyway. That the beauty of an attitude behavior!!!

I just need to do  backups before

 


Best,

iuri

 




*From:* Iuri Sampaio [mailto:[EMAIL PROTECTED]
*Sent:* Wednesday, November 29, 2006 4:38 PM
*To:* 'AOLSERVER@LISTSERV.AOL.COM'
*Subject:* does aolserver run on IBM server

 


Hi,

I’d like to know if it’s possible to install aolserver on a IBM S390 
generation 5 without pain. I have this mainframe and I’m not quite 
sure if it’s safe to have aolserver on it


The link bellow has the specs. About it

http://www-03.ibm.com/servers/eserver/zseries/library/specsheets/pdf/g2214221.pdf
   

 

 


iuri


--
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.5.430 / Virus Database: 268.14.19/556 - Release Date: 
11/28/2006 3:22 PM



--
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.5.430 / Virus Database: 268.15.3/562 - Release Date: 
12/1/2006 1:12 PM


-- AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to 
<[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in 
the email message. You can leave the Subject: field of your email blank.


Sorry for the delayed response. I'm not aware of anyone testing 
AOLserver on the platform you describe, and unfortunately do not have 
access to a similar box to test. I'd be happy to try and help you off 
list if you run into any issues however. Sorry I can't be of more help!


- n


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] errors compiling nsmysql on osx tiger

[Nathan Folkman]

[EMAIL PROTECTED] wrote:
Has anyone been able to compile the nsmysql driver on os x tiger 
10.4.8 / aolserver 4.5 / mysql 5?  I've tried on intel and ppc, 
neither work.


The result of make is as follows:

gcc -pipe -I/usr/local/mysql/include -Os -Wall -Wno-implicit-int 
-fno-strict-aliasing -fno-common  -I/usr/local/aolserver/include 
-I/usr/local/aolserver//include  -DNO_CONST  -DNO_VALUES_H=1 
-DHAVE_LIMITS_H=1 -DHAVE_UNISTD_H=1 -DHAVE_SYS_PARAM_H=1 
-DUSE_THREAD_ALLOC=1 -D_REENTRANT=1 -D_THREAD_SAFE=1 
-DHAVE_PTHREAD_ATTR_SETSTACKSIZE=1 -DHAVE_PTHREAD_ATFORK=1 
-DTCL_THREADS=1 -DHAVE_COREFOUNDATION=1 -DMAC_OSX_TCL=1 
-DTCL_WIDE_INT_TYPE=long\ long -DHAVE_GETCWD=1 -DHAVE_OPENDIR=1 
-DHAVE_STRSTR=1 -DHAVE_STRTOL=1 -DHAVE_STRTOLL=1 -DHAVE_STRTOULL=1 
-DHAVE_TMPNAM=1 -DHAVE_WAITPID=1 -DHAVE_GETPWUID_R_5=1 
-DHAVE_GETPWUID_R=1 -DHAVE_GETPWNAM_R_5=1 -DHAVE_GETPWNAM_R=1 
-DHAVE_GETGRGID_R_5=1 -DHAVE_GETGRGID_R=1 -DHAVE_GETGRNAM_R_5=1 
-DHAVE_GETGRNAM_R=1 -DHAVE_MTSAFE_GETHOSTBYNAME=1 
-DHAVE_MTSAFE_GETHOSTBYADDR=1 -DUSE_TERMIOS=1 -DHAVE_SYS_TIME_H=1 
-DTIME_WITH_SYS_TIME=1 -DHAVE_TM_ZONE=1 -DHAVE_GMTIME_R=1 
-DHAVE_LOCALTIME_R=1 -DHAVE_TM_GMTOFF=1 -DHAVE_ST_BLKSIZE=1 
-DSTDC_HEADERS=1 -DHAVE_SIGNED_CHAR=1 -DHAVE_PUTENV_THAT_COPIES=1 
-DHAVE_LANGINFO=1 -DHAVE_COPYFILE=1 -DHAVE_LIBKERN_OSATOMIC_H=1 
-DHAVE_OSSPINLOCKLOCK=1 -DHAVE_PTHREAD_ATFORK=1 -DUSE_VFORK=1 
-DTCL_DEFAULT_ENCODING=\"utf-8\" -DTCL_LOAD_FROM_MEMORY=1 
-DHAVE_AVAILABILITYMACROS_H=1 -DHAVE_FTS=1 -DHAVE_SYS_IOCTL_H=1 
-DHAVE_SYS_FILIO_H=1  -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" 
-DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" 
-DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 
-DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 
-DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 
-DHAVE_UNISTD_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_TIMEGM=1 -DHAVE_DRAND48=1 
-DHAVE_RANDOM=1 -DHAVE__NSGETENVIRON=1 -DHAVE_GETADDRINFO=1 
-DHAVE_GETNAMEINFO=1-c -o mysql.o mysql.c

mysql.c: In function 'Ns_MySQL_OpenDb':
mysql.c:133: warning: unused variable 'len'
mysql.c:133: warning: unused variable 'y'
mysql.c:133: warning: unused variable 'x'
mysql.c:125: warning: unused variable 'rc'
mysql.c: At top level:
mysql.c:15: warning: 'rcsid' defined but not used
/usr/local/aolserver/bin/tclsh8.4 
/usr/local/aolserver/bin/nsremove.tcl libnsmysql.dylib
gcc -pipe -dynamiclib -install_name 
/usr/local/aolserver/lib/libnsmysql.dylib -flat_namespace -undefined 
suppress  -o libnsmysql.dylib mysql.o   \
 -L/usr/local/aolserver/lib -lnsd -lnsthread 
-L/usr/local/aolserver//lib -ltcl8.4   -lpthread -framework 
CoreFoundation
/usr/local/aolserver/bin/tclsh8.4 
/usr/local/aolserver/bin/nsremove.tcl nsmysql.so
ld -shared -flat_namespace -undefined suppress  -o nsmysql.so  
-R/usr/local/mysql/lib -L. -lnsmysql -L/usr/local/aolserver/lib -lnsd 
-lnsthread -L/usr/local/aolserver//lib -ltcl8.4   -lpthread -framework 
CoreFoundation

ld: unknown flag: -shared
make: *** [nsmysql.so] Error 1


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to 
<[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the 
Subject: field of your email blank.

I have, but I have to run, so I'll post the modified Makefile tomorrow.

- n


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] only one connection thread

[Nathan Folkman]

[EMAIL PROTECTED] wrote:
Following up on Stuart's call for clarification, can someone please send 
some out some documentation, or point us to it, where we can figure out how 
to use the new ns_limits/ns_pools functionality in 4.5.


We just used 4.5 in production and aolserver was limited to 100 ESTABLISHED 
tcp connections, no matter what we tried. We tried to change the old 
settings, changing the keepalivetimeout to 0 and messing with:


ns_param   connsperthread   
ns_param   maxconnections   
ns_param   maxthreads


but not matter what we had in our configuration, nothing helped us get any 
more simultaneous tcp connections.


Any help would be greatly appreciated.

Or, maybe we should just go back to 4.0.10?

Brad


On Thu, 10 Aug 2006 20:59:46 +0100, Stuart Children <[EMAIL PROTECTED]> 
wrote:


  

Stuart Children wrote:


Whilst playing with a new 4.5.0 build, I suddenly noticed that I was
only getting one concurrent connection thread running - eek!
  

OK, so this is an affect of the new limits code. It's more specific than
I initially described - it's one concurrent connection per "specific
url", where I believe url actually means url+at least some headers
(which is slightly surprising, though I can see people having a
requirement for it both ways). Due to the lack of response, I've no idea
whether this is the intentional default behaviour - but it would seem to
merit a mention in the docs if so.

Unfortunately I'm about to leave for an extended holiday, and I've been
pretty busy with other matters in my last few days at work, so
investigating further how the new limits/pools stuff is actually working
(and how one is intended to use it) will have to wait. If anyone can
fill in the gaps whilst I'm away that would be great, but I just wanted
to let people know I probably won't be replying for three weeks...
holiday => no computers. :)

Cheers

--
Stuart


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to 


<[EMAIL PROTECTED]> with the
  
body of "SIGNOFF AOLSERVER" in the email message. You can leave the 


Subject: field of your email blank.
  

=




--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.
  

Any overview of the new commands can be found in the release notes:

   
http://aolserver.cvs.sourceforge.net/aolserver/aolserver/RELEASE?view=markup


Your best bet is to just look at the source:

   aolserver/nsd/limits.c
   aolserver/nsd/pools.c

Here's also a few additional notes I had been working on, although 
probably not that helpful:



 Ns pools


  From Office Wiki

Thread pools can be configured in either application code or the 
AOLserver configuration. The following example will set up a thread pool 
for all GET requests to /slow/* allowing only a maximum of 2 connection 
threads to be allocated. Additional requests will be queued.


ns_pools set "slow" -maxthreads 2
ns_pools register "slow" "server1" "GET" "/slow/"

There are a few stats commands available as well:

server1:nscp 1> ns_server threads slow
{min 0} {max 2} {current 2} {idle 0} {stopping 0}

server1:nscp 2> ns_pools list
slow default error

server1:nscp 3> ns_pools get slow



 Ns limits


  From Office Wiki

In addition to creating and registering thread pools, you can also 
specify the following limits:


maxrun maxwait maxupload timeout

minthreads 0 maxthreads 2 idle 0 current 2 maxconns 2 queued 8 timeout 5


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] gzip output compression

[Nathan Folkman]

[EMAIL PROTECTED] wrote:
One of the New adp improvement of aolserver 4.5 is   gzip output 
compression. .

Is there any documentation for using it ?
Do i need to define any header to enable it ?
how do  i enable it and check if its working ?


Thanks



Hi, you can take a look at some of the included documentation for the 
new "nszlib" module:


http://aolserver.cvs.sourceforge.net/aolserver/aolserver/nszlib/README?view=markup
http://aolserver.cvs.sourceforge.net/aolserver/aolserver/nszlib/example.tcl?view=markup

Hope that helps!

- n


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] Google's TCMalloc

[Nathan Folkman]

[EMAIL PROTECTED] wrote:
Can anyone guide me on how to compile Google's TCMalloc with aolserver 
i would like to replace it with TCL threaded memory allocator  ?

This is in reference to http://webservertalk.com/message1631310.html


thanks






You'll need to build Tcl so that it doesn't use the threaded allocator, 
then you should be able to LD_PRELOAD TCMalloc per the Google 
instructions. Building Tcl threaded, but so that it doesn't use the 
threaded allocator used to require a hack of sorts - not sure if this 
has been fixed in the latest versions or not.


- n


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] Tcl interpreter bloating

[Nathan Folkman]

[EMAIL PROTECTED] wrote:

This morning my server complained:
"unable to alloc 2169111 bytes"

How large was the nsd process when it crashed?


and thus exited.  So, I'm thinking that malloc fragmentation may very 
well be the problem.


I read recently that the
connsperthread
parameter no longer works in aolserver.

I believe this parameter destroys a Tcl interpreter and recreates it, 
yes?.


This technique seems to me a much better way to deal with tcl memory 
fragmentation than experimenting with a different malloc (ie, it's 
parallel to the way Apache deals with memory bloat in mod_perl).


I'm trying (right now on the live server) to do a manual version of 
connsperthread, using ns_markfordelete and a per-interpreter counter, 
cleaning up an interpreter after 1000 uses.  I'll be curious to see if 
that fixes the bloat problem.


-john


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to 
<[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the 
Subject: field of your email blank.



--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] Tcl interpreter bloating

[Nathan Folkman]

[EMAIL PROTECTED] wrote:


So did setting connsperthread take care of the bloating? I keep 
seeing assertions that memory use by AOLserver should level off at 
some steady state - perhaps controlled by this parameter. But I have 
yet to see any one come back and say "I changed X to Y and now my 
server doesn't behave like it has a memory leak".


FYI, BookMooch.com bloats to 2gigs within 4 days of use and needs to 
be restarted.


I'm hoping that the problem is with the berkeleydb driver AOLserver 
driver, and am going to dry another database connectivity technique 
(RMI) to see if that fixes it.  If not...


-john


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to 
<[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the 
Subject: field of your email blank.
Your problem is most likely the Tcl memory allocator, and fragmentation 
that can occur within the pools. There have been some posts detailing 
these issues a few months ago I think.


One thing to try is building Tcl so that it doesn't use the 
threaded-allocator, and to instead try something like Hoard or Google's 
tcmalloc. Now all that aside, is it really a big deal to do rolling 
restarts?


- n


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] More config questions

[Nathan Folkman]

[EMAIL PROTECTED] wrote:

On 9/12/06, Nathan Folkman <[EMAIL PROTECTED]> wrote:

They are still supported. The "base.tcl" was supposed to be a simpler
and more bare bones configuration example. The plan was to add other
smaller configuration examples, such as the "nscp.tcl" which could then
be combined with the "base.tcl." Never finished - sorry!


I just did a check of the 4.5 code and I can't find anywhere where it
is reading maxthreads or minthreads from the config.tcl.   Changing
any of these settings has no effect.

ns_section "ns/server/server1"
ns_param maxthreads 20
ns_param minthreads 5
ns_param maxconns 20

Danie

These options  have moved to the new "ns_pools" interface:

[1153ahost147:~/cvs/aolserver_head/nsd] shmooved% grep maxconns *
...
pools.c:"-maxthreads", "-minthreads", "-maxconns", "-timeout", NULL
...

- n


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] More config questions

[Nathan Folkman]

[EMAIL PROTECTED] wrote:

I think my last message on this was eaten...

Looking at the demo config (base.tcl) and the code for 4.5, there are
a lot of options in the config that are no longer supported by code.
In earlier version you could start up a defined number of threads
(minthreads, maxthreads and maxcons) in the config.   Now those
options seem to be gone and on my particular install all I see is a
single thread handing everything?  Is this right?  Is this efficient?

Daniel

They are still supported. The "base.tcl" was supposed to be a simpler 
and more bare bones configuration example. The plan was to add other 
smaller configuration examples, such as the "nscp.tcl" which could then 
be combined with the "base.tcl." Never finished - sorry!


- n


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] internal server redirects

[Nathan Folkman]

The segfault is obviously a bug... I think what's being referring to is 
the behavior that allows code to continue executing after an 
"ns_returnredirect" unless an "ns_adp_abort" is explicitly specified.


- n

[EMAIL PROTECTED] wrote:
And the segfaulting when you do this?  Also a feature?  I'd call it a 
bug.  Nothing you can do in an adp, scheduled proc, etc, etc (plain 
tcl) should cause the server to crash.


Rusty

Titi Ala'ilima wrote:
It's not a bug, it's a feature.  Seriously.  Sometimes you want to 
return a response to the connection and then continue processing 
afterwards.  Or at least, sometimes _I_ want to do that. :-)


-Titi

-Original Message-
From: Daniel P. Stasinski [mailto:[EMAIL PROTECTED] Sent: 
Wednesday, September 06, 2006 2:10 PM

To: AOLSERVER@LISTSERV.AOL.COM
Subject: Re: [AOLSERVER] internal server redirects



it still crashes.


Ok, I just found this on the new docs

NOTE: The script does not end at the time this command is invoked. 
ns_adp_abort or ns_adp_return should be called to end script 
processing after ns_returnredirect


This does stop the crash but I would still consider it to be a bug.

Daniel


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to 
<[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the 
Subject: field of your email blank.



--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to 
<[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the 
Subject: field of your email blank.



--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] AOLserver's documentation woes and its future

[Nathan Folkman]

[EMAIL PROTECTED] wrote:

Dossy Shiobara wrote:


It's about why we end up in situations where we even feel a debugger is
necessary.


I used to use debuggers much more than I do currently, (probably in 
large part because I used to do more C coding), but even though I do 
99% of my debugging without a debugger, that remaining 1% would be far 
more painful without the debugger.


There are 2 main places where I find a debugger helpful.  One is when 
I make stupid mistakes (and if you don't occasionally make stupid 
mistakes then I doubt you're actually programming) and the debugger 
can smack me upside the head and point out the stupidity ("Oh, it's a 
fencepost error!").   The other, more important case is when I am 
digging into code that I am not familiar with.  In those cases, the 
debugger can give a new perspective and let you realize what is going 
on in the code, at which point you can ditch the debugger and go back 
to the code with a better understanding.  Once you are familiar with 
the code then the debugger becomes much less useful, but in those 
initial stages it greatly speeds the progress.


Linus's rant really rubbed me the wrong way.  I thought "considered 
harmful" essays went out of vogue a few decades ago.  If you'd rather 
not use a particular tool fine, but don't imply that people who choose 
to use it are somehow inferior.


I thought there was code in AOLserver to support the TclPro debugger - 
has that been removed?

The hooks still exist and it even still works.


 Debugging AOLserver pages

   * Configure AOLserver to allow debugging, and restart:

ns_section "ns/server/server1/adp"
   ns_param enabledebug true

   * Download and install the TclDevKit

   * Start the debugger:

% tcldebugger &

   * Start a new session:

File > New-Project

   * Start a new remote session:

Debugging Type > Remote Debugging

   * Append the following information to the page you wish to debug:

?debug=&dhost=&dport=&dprocs=

   * AOLserver Dubugging Query Arguments:

debug   String. The files you wish to debug. (Example: *, *.adp, foo.inc)
dhost   String. The host name the Tcl debugger is listening on. (Example: 
127.0.0.1)
dport   Integer. The port the Tcl debugger is listening on. (Example: 2576)
dprocs  String. Procedures to instrument. (Example: ns*, foo_something)

   * Example:

http://foo.com:8000/test.adp?debug=*&dhost=127.0.0.1&dport=2576 


Hope that helps!

- n 







-J


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to 
<[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the 
Subject: field of your email blank.



--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] AOLserver's documentation woes and its future

[Nathan Folkman]

[EMAIL PROTECTED] wrote:

On 5 Sep 2006, at 17:57, Titi Ala'ilima wrote:
1) AOLserver probably needs a new name.  Something that uses the NS 
initials would be ideal so that all those ns_* commands actually make 
sense again.  Could we resurrect the NaviServer name?
Already taken! (OpenSource fork-without-anyone-from-AOL-interfering of 
AOLserver)
Maybe we should all focus on NaviServer going forward? Those guys have 
done a good job backporting work that's been done to the AOLserver head. 
Just a thought.


- n


Bas.


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to 
<[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the 
Subject: field of your email blank.



--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] AOLserver's documentation woes and its future

[Nathan Folkman]

[EMAIL PROTECTED] wrote:

(Apologies for the long email ahead ... but, I think it's worth a quick
read.)

On 2006.09.05, Rick Gutleber <[EMAIL PROTECTED]> wrote:
  
Support for more popular languages (come on, let's say it together, I 
know it's hard, but "Tcl is not popular") is probably the most useful 
long-term technical change that can be made.



Of course, the more popular languages weren't implemented with
embeddability (embedibility?) in mind.  Tcl is one of the only mature
languages that can be safely embedded in a multi-threaded application.

Specifically, what I mean is, Tcl is one of a few languages that can
have multiple interpreters in the same process space executing
concurrently in multiple threads.

Python can be embedded, but according to this:

http://docs.python.org/api/api.html

8.1 Thread State and the Global Interpreter Lock
http://docs.python.org/api/threads.html

"The Python interpreter is not fully thread safe. In order to
support multi-threaded Python programs, there's a global lock that
must be held by the current thread before it can safely access
Python objects. [...]"

A global lock?  There goes scalability and the benefits of embedding
Python.  (You'd be better off running it in a separate process ala
FastCGI, AOLserver would gain no performance by embedding it, only
risk.)

Same problem with Ruby:

http://metaeditor.sourceforge.net/embed/

"ruby is not thread-safe. If you embed ruby into a multi-threaded
application then you will need to semaphore protect all access to
ruby. Warning: Insufficient semaphore protection can be hard
debuggable."

So, they're pointing out the need for a global lock, too.  What's worse,
the Ruby interpreter uses global variables to manage interpreter state,
so even if you did embed it and use a global lock, you couldn't spin up
two instances of the interpreter in the same process, anyway!

PHP isn't even worth considering, really:

Re: [PHP-DEV] Re: PHP and Apache2
http://marc.theaimsgroup.com/?l=php-dev&m=108736540021355&w=2

"The major feature that draws people to Apache2 is threading. [...]
However, on UNIX there are a lot of basic libraries where thread
safety is an unknown.  [...]

And yes, you could use the prefork mpm with Apache2 to avoid the

threading, and yes you could use a standalone fastcgi mechanism to
avoid the threading, but you are going out of your way to avoid the
defining characteristic of the web server you have decided to use."

Rasmus makes it very clear that the reason PHP is so popular and
powerful is its large list of third-party extensions that add a lot of
"out of the box" functionality for PHP developers.  However, it's that
same attribute of PHP that keeps it from playing nicely, embedded in a
multi-threaded application.

Lets not forget this, too:


http://developers.slashdot.org/developers/06/07/28/0552240.shtml

"Jani Taskinen, one of the lead developers of the Zend Engine (the
engine that powers PHP), as well as a lead developer for the thread
safety system and other core components of the PHP project, has quit
in a relatively cryptic message to the php-internals mailing list.
Jani has been involved with PHP for about 6 years and his loss will
undoubtedly be a big blow for the PHP project."

It'll be interesting to see where the future of PHP takes it.

Now, lets look at Perl:

http://search.cpan.org/dist/perl/pod/perlembed.pod

"Now suppose we have more than one interpreter instance running at
the same time. This is feasible, but only if you used the Configure
option -Dusemultiplicity or the options -Dusethreads -Duseithreads
when building perl. [...]

Using -Dusethreads -Duseithreads rather than -Dusemultiplicity is
more appropriate if you intend to run multiple interpreters
concurrently in different threads, because it enables support for
linking in the thread libraries of your system with the
interpreter."

Oh, nice.  I don't see any mention of using a globally shared lock to
prevent two Perl interpreters running concurrently.  (This could just be
an omission from the docs, but I have the feeling it's not the case.)

So, of the "languages more popular than Perl" being Python, Ruby, PHP
and Perl ... Perl is the only suitable candidate for replacing Tcl in
AOLserver.

Considering that Perl is no longer the "flavor of the day" and has been
replaced by Python and Ruby amongst the "language geeks" and PHP amongst
the unwashed masses ... who really wants to go replacing AOLserver's
embedded Tcl with embedded Perl?  How many months or years before people
see Perl like they see Tcl today (not popular, etc.)?

The irony here is so few people have even heard of Tcl that if it was
evangelized and marketed correctly, it could be brought to the
marketplace as a "new" language to replace Ruby and Python and PHP.
Write a slew of books, release a mound of free 

Re: [AOLSERVER] Is tcl exec _really_ bad with threads

[Nathan Folkman]

[EMAIL PROTECTED] wrote:

On Aug 30, 2006, at 10:32 AM, Nathan Folkman wrote:

So the easy way to do this would be to exec from the nsproxy. Better 
option might be to wrap the C API, although I'm not entirely sure 
whether or not Imagemagick is truly thread-safe. Even if it is not, 
you'd probably still see better performance using the wrapped C API 
from the nsproxy.


Is this concern something new for 4.5?  Because OpenACS has been 
exec'ing Imagemagik, sqlplus and plsql for years now with no apparent 
problems.  Have we just been really lucky?


janine


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to 
<[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the 
Subject: field of your email blank.
If it ain't broke don't fix it. ;-) But if you're running into memory 
issues, the nsproxy approach I mentioned can help. Another nice thing in 
general about running code from within the nsproxy is that you can 
easily kill of the nsproxy processes which can help you reclaim memory 
that otherwise might be tied up in the Tcl threaded allocator shared 
pools. This is more of a general side affect, and not really relevant to 
the exec'ing discussion...


- n


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] Is tcl exec _really_ bad with threads

[Nathan Folkman]

[EMAIL PROTECTED] wrote:

On Wednesday 30 August 2006 10:32, Nathan Folkman wrote:
  

[EMAIL PROTECTED] wrote:


On Aug 30, 2006, at 8:56 AM, Nathan Folkman wrote:
  

What is it you are trying to exec?


From OpenACS we mostly just exec Imagemagick.  I can't think of
  


  

So the easy way to do this would be to exec from the nsproxy. Better
option might be to wrap the C API, although I'm not entirely sure
whether or not Imagemagick is truly thread-safe. Even if it is not,
you'd probably still see better performance using the wrapped C API from
the nsproxy.




I've loaded imagemagick into AOLserver: it was much slower than exec'ing the 
command line tools. 


tom jackson


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.
  
Wow, that's surprising! Then exec'ing from the nsproxy is probably your 
best bet.


- n


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] Is tcl exec _really_ bad with threads

[Nathan Folkman]

[EMAIL PROTECTED] wrote:

On Aug 30, 2006, at 8:56 AM, Nathan Folkman wrote:


What is it you are trying to exec?


From OpenACS we mostly just exec Imagemagick.  I can't think of 
anything else.


janine


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to 
<[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the 
Subject: field of your email blank.
So the easy way to do this would be to exec from the nsproxy. Better 
option might be to wrap the C API, although I'm not entirely sure 
whether or not Imagemagick is truly thread-safe. Even if it is not, 
you'd probably still see better performance using the wrapped C API from 
the nsproxy.


- n


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] Is tcl exec _really_ bad with threads

[Nathan Folkman]

[EMAIL PROTECTED] wrote:

On Tuesday 29 August 2006 20:29, Jeff Hobbs wrote:
  

You are correct that the main problem is in trying to fork when not
closely followed by exec*().



But with Tcl exec, you never call fork, screw around and then call exec*(), do 
you? Is this an issue for C level tcl? Maybe the issue is that fork and exec 
are separate calls, and something unplanned could happen between the two, and 
pthread_atfork makes the two calls atomic. (And maybe sometimes even this 
isn't enough?)


Obviously we can now use nsproxy to call exec. I'm not sure if it works for 
every use of exec inside a full AOLserver, but it would be pretty close.


tom jackson


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.
  

What is it you are trying to exec?

- n


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] Is tcl exec _really_ bad with threads

[Nathan Folkman]

Just saw this on Digg - pretty hip!

http://www.cit.gu.edu.au/~anthony/graphics/imagick6/

- n

[EMAIL PROTECTED] wrote:
An alternate strategy to exec'ing is to instead expose the low-level 
C/C++ APIs as Tcl commands. If the code in question isn't thread safe, 
you can create a library that you would load from the nsproxy, which 
if you remember, is basically just a single-threaded tclsh that is 
fork'd and exec'd from the main nsd process.


   http://www.imagemagick.org/www/magick-wand.html

- n

[EMAIL PROTECTED] wrote:
I use exec for Image Magick, though not in a high-concurrency 
environment.
I wonder how Photo.net deals with this; I always understood they use 
Image
Magick this way. Maybe they did in the beginning and have now 
switched to

a module?

Does anyone have experience with using TclMagick inside AOLserver?

Cheers,
Bas.


On Wednesday, August 30, 2006 4:29, Jeff Hobbs said:
 

Tom Jackson wrote:
   

On Saturday 19 August 2006 21:07, Tom Jackson wrote:
 

On Saturday 19 August 2006 19:12, Hossein Sharifi wrote:
   

 (although I do plan to fix the
incorrect usage of exec as well)
  
So I've never heard that you can't use exec from AOLserver. How is 
this
supposed to be done? Where was this discussed? What would an error 
look

like?


Sorry to respond to my own last post, but I've been trying to find out
more
about this problem of using tcl exec.

As far as I can tell, the problem is trying to use fork (in some tcl
extension
that adds this) and not calling exec in the child process. Is this 
even

possible in exec to not call exec?

Does anyone know the status of this problem in the tcl community, from
what I
have read, the solutions would not be good for AOLserver.

The best rundown, with links, is here:


  

Haven't written that thread, what else more specifically do you want to
know?  ActiveTcl has been shipping thread-enabled for the last several
releases.  There are some issues with non-core packages (and Tk itself,
but more recently resolved), but it has worked well in general.  OS X
also is thread-built, and even has pthread_atfork usage on Tiger, 
but it

is slightly prone to issues, and OS X has a different fork()
implementation than some other common unix variants.

You are correct that the main problem is in trying to fork when not
closely followed by exec*().

Jeff


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to
<[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the
Subject: field of your email blank.





--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to 
<[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the 
Subject: field of your email blank.
  





--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] aolserver tuning thru nstelemetry ?

[Nathan Folkman]

Couple of general things about the Tcl threaded memory allocator (zippy) 
that might help better explain what you are seeing...


- The Tcl memory allocator is a x2 allocator optimized for lock 
avoidance between threads. This results in very high performance, but 
comes at the cost of almost x3 overhead for allocations.


- There is no garbage collection mechanism provided by the Tcl threaded 
allocator for pool allocations. Once memory is allocated, it flows back 
and forth between thread memory pools and the shared memory pool. The 
exception are very large allocations (the exact threshold escapes me at 
the moment.) These large allocations are mmap'd directly, and free'd 
back to the system.


- Over extended runs, the memory can tend to get fragmented to the point 
where the allocator is unable to effectively coalesce smaller blocks 
together, causing additional memory to be allocated from the system.


- In general memory stays tied up in per-thread caches unless thread 
cleanup occurs, at which point the memory is returned to the shared 
pool. So if you have a large number of threads which do things like sit 
in "while 1" loops, you could be holding on to large amounts of memory 
depending on any number of factors.


- If you have third-party code which isn't using Tcl_Alloc() and 
Tcl_Free(), then you aren't using the Tcl threaded allocator.


We actually spent a great deal of time trying to tweak the allocator to 
work around these issues, but in the end the results were inconclusive, 
and we abandoned this work:


   http://aolserver.cvs.sourceforge.net/aolserver/zippy2/

  22 /*
  23  * Design summary:
  24  *
  25  * This allocator is based on the legacy single-threaded Tcl allocator
  26  * with modifications necessary to support additional features:
  27  *
  28  * 1. Avoid lock contention:  Storage requests smaller than a system
  29  * page are allocated with BinAlloc which returns pre-allocated blocks
  30  * from per-thread caches.  Most small requests can thus be satisfied
  31  * without acquiring or releasing mutex locks. To moderate overall
  32  * memory growth, bins have a high water mark which defaults to 2x the
  33  * number of blocks per system page.  When available blocks exceed this
  34  * limit, half of the free blocks are moved to a separate shared cache.
  35  * When no blocks are available in a per-thread cache, threads first
  36  * check the shared cache for these free blocks before calling PageAlloc
  37  * to allocate a new page.  All free blocks are moved to the shared
  38  * cache before a thread exists.  In this way, a long running process
  39  * should see continual movement of small blocks between the per-thread
  40  * and shared caches.
  41  *
  42  * 2. System allocation for large storage requests: The allocator has a
  43  * configurable storage size threshold over which allocations are
  44  * performed with system-allocation based MapAlloc and MapFree direclty,
  45  * bypassing the malloc-based routines. This has the benefit of
  46  * immediately returning to the system very large storage requests,
  47  * avoiding the possibility the malloc-based routines holding onto the
  48  * large regions. Many malloc-based implementations already support this
  49  * operation for some or most allocations so this capability may be
  50  * redundant with your particular system.
  51  *
  52  * 3. Ability to return unused blocks to the system:  Cached blocks are
  53  * obtained by splitting system pages allocated with PageAlloc.  The
  54  * "alloc reclaim" command can be used to sort blocks by their
  55  * cooresponding pages and returned complete pages to the free list.
  56  * These pages can then be returned to the system with the "alloc
  57  * freepages" command.  A more aggressive reclaim is possible with the
  58  * "alloc wreclaim" command which will force other threads which call
  59  * allocation routines to dump all available blocks and wait for the
  60  * reclaim operation to complete.
  61  *
  62  * 4. Activity monitoring: For all storage requests, even those beyond
  63  * the small blocks in per-thread caches, numerous counters are
  64  * maintained on per-thread and a summary basis.  The shared cache is
  65  * used to accumulate counters for all threads which have previously
  66  * exited.  In this way, the current and past state of all allocations
  67  * for can be summarized assuming the various counters do not overflow
  68  * in the course of a long running process. These counters are available
  69  * through the alloc "binstats", "slotslots", and "info" options or the
  70  * the Tcl_GetMemoryInfo procedure.
  71  *
  72  * 5. Tcl_Obj allocation: Allocations for Tcl_Obj's are handled as a
  73  * special case, returning the memory without the normal rounding to the
  74  * nearest block size and without a preceding Block structure.
  75  *
  76  * 6. Tunable parameters: Environment variables can be used to tune the
  77  * sizes and

Re: [AOLSERVER] Is tcl exec _really_ bad with threads

[Nathan Folkman]

An alternate strategy to exec'ing is to instead expose the low-level 
C/C++ APIs as Tcl commands. If the code in question isn't thread safe, 
you can create a library that you would load from the nsproxy, which if 
you remember, is basically just a single-threaded tclsh that is fork'd 
and exec'd from the main nsd process.


   http://www.imagemagick.org/www/magick-wand.html

- n

[EMAIL PROTECTED] wrote:

I use exec for Image Magick, though not in a high-concurrency environment.
I wonder how Photo.net deals with this; I always understood they use Image
Magick this way. Maybe they did in the beginning and have now switched to
a module?

Does anyone have experience with using TclMagick inside AOLserver?

Cheers,
Bas.


On Wednesday, August 30, 2006 4:29, Jeff Hobbs said:
  

Tom Jackson wrote:


On Saturday 19 August 2006 21:07, Tom Jackson wrote:
  

On Saturday 19 August 2006 19:12, Hossein Sharifi wrote:


 (although I do plan to fix the
incorrect usage of exec as well)
  

So I've never heard that you can't use exec from AOLserver. How is this
supposed to be done? Where was this discussed? What would an error look
like?


Sorry to respond to my own last post, but I've been trying to find out
more
about this problem of using tcl exec.

As far as I can tell, the problem is trying to use fork (in some tcl
extension
that adds this) and not calling exec in the child process. Is this even
possible in exec to not call exec?

Does anyone know the status of this problem in the tcl community, from
what I
have read, the solutions would not be good for AOLserver.

The best rundown, with links, is here:


  

Haven't written that thread, what else more specifically do you want to
know?  ActiveTcl has been shipping thread-enabled for the last several
releases.  There are some issues with non-core packages (and Tk itself,
but more recently resolved), but it has worked well in general.  OS X
also is thread-built, and even has pthread_atfork usage on Tiger, but it
is slightly prone to issues, and OS X has a different fork()
implementation than some other common unix variants.

You are correct that the main problem is in trying to fork when not
closely followed by exec*().

Jeff


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to
<[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the
Subject: field of your email blank.





--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.
  



--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] Is tcl exec _really_ bad with threads

[Nathan Folkman]

[EMAIL PROTECTED] wrote:

On Saturday 19 August 2006 21:07, Tom Jackson wrote:
  

On Saturday 19 August 2006 19:12, Hossein Sharifi wrote:


 (although I do plan to fix the
incorrect usage of exec as well)
  

So I've never heard that you can't use exec from AOLserver. How is this
supposed to be done? Where was this discussed? What would an error look
like?



Sorry to respond to my own last post, but I've been trying to find out more 
about this problem of using tcl exec. 

As far as I can tell, the problem is trying to use fork (in some tcl extension 
that adds this) and not calling exec in the child process. Is this even 
possible in exec to not call exec?


Does anyone know the status of this problem in the tcl community, from what I 
have read, the solutions would not be good for AOLserver.


The best rundown, with links, is here:



tom jackson


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.
  
Something else to watch out for is the side effect of doing a fork() 
then exec() from the main nsd process. There's a small window of time 
where you can have double the memory of your main nsd process tied up 
while the main nsd process space is copied. So, for example, if your nsd 
runs really hot at say 1G, then you'd need at least an extra 1G, and 
maybe more if you have multiple threads all trying to fork() and exec() 
something. We've seen this with things like the nsdb code and the new 
nsproxy code. Generally this isn't a problem, but something to be aware of.


That said, a much better strategy is to immediately fork off an nsproxy 
and do all further execs from within the nsproxy (tclsh) which is 
considerably smaller then the main nsd process.


Just curious, what is it that you are looking to exec?

- n

- n


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] XOTlc and AOLServer 4.5

[Nathan Folkman]

[EMAIL PROTECTED] wrote:

Hi!

I'm trying to use the throttle xotcl package with AOLServer 4.5, which works 
in 4.0.10. When I call a page, I get the error 


invalid command name "::xotcl::Object"

from the 


ns_register_filter postauth GET * throttle

page filter.

The xotcl namespace is imported in a tcl library file at startup, I even tried 
to import it with ns_eval.


Has something changed in the way AOLServer imports namespaces at startup?

Thanks,

wiwo


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.
  

Try adding the following to your Tcl library which gets sourced at startup:

ns_ictl package require xotcl

See also: 
http://aolserver.cvs.sourceforge.net/aolserver/aolserver/doc/ns_ictl.n?view=markup


- n


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] cron (ie, timed) jobs in aolserver

[Nathan Folkman]

Have you looked at the "ns_schedule_proc" APIs?

Example:

    http://panoptic.com/wiki/aolserver/Ns_schedule_daily

- n

 

-Original Message-
From: [EMAIL PROTECTED]
To: AOLSERVER@LISTSERV.AOL.COM
Sent: Tue, 15 Aug 2006 6:48 PM
Subject: [AOLSERVER] cron (ie, timed) jobs in aolserver









Anyone have any advice on how to run tasks on a scheduled basis within the aolserver process? 
 

Currently, I use cron to run "lynx http://bookmooch.com/nightlyjobs.adp", and that works fine, but was wondering if there was an all-aolserver way, that was more elegant. 
 

-john 
 


-- 

AOLserver - http://www.aolserver.com/ 
 

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the 

body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank. 



 




Check out AOL.com today. Breaking news, video search, pictures, email and IM. All on demand. Always Free.





--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.

Re: [AOLSERVER] question about libnsd loading in tclsh

[Nathan Folkman]

Locks are not shared with the nsd process. The libnsd library came about as a result of folks wanting some of the AOLserver core Tcl APIs from within stand alone tclsh processes.

You might also want to look at the new nsproxy module. That code allows you to start a pool of tclsh processes that you can send commands to. It's a great way to deal with code which is not thread safe, since each tclsh, by default, is single threaded. The tclsh processes are fork'd and exec'd from the main nsd process.

Example:

    ns_proxy pool
   -
nsd (MT) <> tclsh (ST)
 <> tclsh (ST)
 <> tclsh (ST)

- n


 
 
-Original Message-
From: [EMAIL PROTECTED]
To: AOLSERVER@LISTSERV.AOL.COM
Sent: Tue, 15 Aug 2006 5:52 PM
Subject: [AOLSERVER] question about libnsd loading in tclsh









In the 4.5 release docs, it was announced: 
 

libnsd: 

    The AOLserver library now includes an entry point suitable 

    for loading into an ordinary, thread-enabled, tclsh, e.g.,: 
 

    # tclsh 

    % load /usr/local/aolserver/lib/libnsd.so 

    % ns_time 
 

Does this loaded libnsd.so share locks, such as ndb_handles and semaphores with an already running nsd? 
 

I assume not, but thought I'd ask... 
 

What I'm trying to do is run command-line tcl commands that would access my berkeleydb database while aolserver is still running, and if two separate processes are writing the same db file, it gets corrupted. 
 

-john 
 


-- 

AOLserver - http://www.aolserver.com/ 
 

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the 

body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank. 



 




Check out AOL.com today. Breaking news, video search, pictures, email and IM. All on demand. Always Free.





--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.

Re: [AOLSERVER] cvs head build problem

[Nathan Folkman]

The plan was to roll all that back so we could get to a simple, and some 
would argue, more standard build style:


- autoconf
- configure
- make

Not quite there yet though it seems... In the mean time, the README file 
is accurate, and contains instructions that illustrate how things differ 
from how the 4.0 and earlier versions are built.


- n

[EMAIL PROTECTED] wrote:

Actually was configure changed back to being the main build?
Last time I played with it spent half a night writing a patch until 
Nate told me

to look at the Readme :-)

It was changed to doing
% /path/to/tclsh nsconfig.tcl

Which then calls configure- or has it been updated?
P

Dossy Shiobara wrote:

On 2006.08.07, John Buckman <[EMAIL PROTECTED]> wrote:
  
The modifications to util/*.tcl stated above seem critical and the  
CVS head should be so patched.



The build is still kinda shaky.  In theory, you could build like this
(on Debian):

$ ./configure TCLSH=/usr/bin/tclsh --with-tcl=/usr/lib/tcl8.4 
$ make

$ make install

And it'll use the path specified in TCLSH as the executable to use to
execute the .tcl scripts.

I'm not sure I like this: for one, we should probably use autoconf to
auto-detect the tclsh binary using the TCL_EXEC_PREFIX from tclConfig.sh
or some other magic as a default.  That would probably eliminate the
most FAQ build issue with 4.5.0 that you ran up against.

The pre-3.3 gcc error you encountered I created a patch for and folks
have verified it fixes the problem, so I'll commit that now.

Thanks again for the feedback and congratulations on BookMooch.com, by
the way.  I love the Server: header.  :-)

-- Dossy

  


--
Patrick O'Leary

AOL Search
Phone: + 1 703 265 8763

Honesty is the best policy, but insanity is a better defense !
  


-- AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to 
<[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in 
the email message. You can leave the Subject: field of your email blank.





--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] Fedora 5 and aolserver 4.5.0

[Nathan Folkman]

Try the following steps (if you haven't already) from the aolserver 
directory, and see if this works for you:


1. ./configure --with-tcl=/usr/local/tcl/lib 
--prefix=/usr/local/aolserver --enable-symbols
2. /usr/local/tcl/bin/tclsh8.4 nsconfig.tcl -install 
/usr/local/aolserver -debug

3. make
4. make install

- n

[EMAIL PROTECTED] wrote:

I could never make 4.0.10 work with FC5 even with a local version of tcl.

multiple definition of `_init'

shows up no matter what I do.

I am now trying to compile 4.5 and the following error is thrown.

nsdb nssock nslog nsperm nscgi nscp
make: execvp: ./util/nsmakeall.tcl: Permission denied
make: *** [build] Error 127

Any ideas?
It appears that Fedora and AOL don't like each other. All my other 
boxes are gentoo and I have no problems.



--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to 
<[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the 
Subject: field of your email blank.



--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] [ANN] AOLserver 4.5.0 released!

[Nathan Folkman]

I'd like to say categorically "yes", but it depends on specifically 
which issues you were referring to. There have been some 
"misconceptions" in the past about AOLserver's memory usage patterns in 
regards to how the threaded allocator works, and I'm aware of a few 
leaks in some older versions of Tcl, but to the best of my knowledge 
these have been addressed in the current Tcl 8.4.13 release. Was there a 
specific issue you were concerned about?


- n

[EMAIL PROTECTED] wrote:
There were a number of discussions re AOLserver 4.10 memory usage 
problems, which were associated with at least one version of Tcl.


Is it fair to assume that those memory usage problems have been 
eliminated in AOLserver 4.5.0 when used with Tcl 8.4.1.3?


Thanks.

Harvey Lunenfeld
Internet Corporation
[EMAIL PROTECTED]



- Original Message - From: "Nathan Folkman" <[EMAIL PROTECTED]>
To: 
Sent: Thursday, July 06, 2006 1:50 PM
Subject: Re: [AOLSERVER] [ANN] AOLserver 4.5.0 released!


The latest Tcl version (8.4.13) works great! Haven't spent any amount 
of time testing with upcoming Tcl 8.5, but it also should work fine I 
believe.


Not sure exactly what memory issues you are referring to, but if you 
are building Tcl multi-threaded, then you are using the threaded 
allocator (former Zippy allocator) which is simple x2 allocator. The 
allocator is optimized for front-end performance, and delivers pretty 
much 0% lock contention. This comes at the cost of wasted bytes 
through bucket size allocations, and also there is currently no 
garbage collection facilities provided.


We recently released a new version of the Tcl memory allocator code 
which does provide garbage collection, but it was unclear in our 
testing if that approach would ever really be valid. If high 
performance is your goal, then the Tcl threaded allocator is probably 
best for you. Alternately you could try something like Hoard, or even 
Google's tcmalloc.


Hope that helps!

- n

[EMAIL PROTECTED] wrote:

What version of Tcl is recommended for use with AOLserver 4.5.0?

Have memory usage/depletion issues been resolved in AOLserver 4.5.0, 
when

using the recommended version of Tcl?

Harvey Lunenfeld
Internet Corporation
[EMAIL PROTECTED]





- Original Message - From: "Dossy Shiobara" 
<[EMAIL PROTECTED]>

To: 
Sent: Tuesday, June 27, 2006 3:55 PM
Subject: [AOLSERVER] [ANN] AOLserver 4.5.0 released!



AOLserver 4.5.0 Released
June 27, 2006


   On behalf of the AOLserver Team, I have the honor of announcing the
   latest major release of AOLserver: 4.5.0.

   AOLserver 4.5.0 is a major upgrade including several new Tcl 
commands,

   C API's, and resource management improvements.

   A summary of changes is provided at the end of this document.


WHAT IS AOLSERVER?

   AOLserver is America Online's Open-Source web server.  AOLserver is
   the backbone of the largest and busiest production environments in
   the world.  AOLserver is a multithreaded, Tcl-enabled web server
   used for large scale, dynamic web sites.

   Visit the project's website:

   http://aolserver.com/


HOW CAN I GET AOLSERVER?

   Download the source code from SourceForge:

   http://aolserver.com/downloads/aolserver-4.5.0-src.tar.gz


I FOUND A BUG!  WHAT DO I DO?

   File it in the Bug Tracker at SourceForge:

   http://aolserver.com/sf/bugs


IS THERE A MAILING LIST?  WHERE'S THE USER COMMUNITY?

   Yes!  There's an announcements-only and a general discussion 
mailing

   list.  Instructions on how to subscribe are here:

   http://aolserver.com/lists.php

   There is also a wiki-web set up for AOLserver:

   http://aolserver.com/wiki/


CHANGES SINCE LAST RELEASE?

AOLserver 4.5 - June 2006
-

AOLserver 4.5 is a major upgrade including several new Tcl commands,
C API's, and resource management improvements.


Contents:


New and Updated Binaries
New Modules
New and Updated Tcl Commands
New and Updated C Routines
Additional Updates
Changes to the Build Process
Bug Fixes
Known Issues


New and Updated Binaries:


libnsd:
   The AOLserver library now includes an entry point suitable
   for loading into an ordinary, thread-enabled, tclsh, e.g.,:

   # tclsh
   % load /usr/local/aolserver/lib/libnsd.so
   % ns_time

   This provides the full set of Tcl commands within the
   AOLserver core although several commands will generate
   errors for lack of an HTTP request context (e.g., ns_conn).

nstclsh:
   A simple Tcl-shell linked against libnsd. It's equivalent
   to loading libnsd into an ordinary tclsh as described above.


New Modules:
---

nszlib:
   The popular nszlib module written by Vlad Seryakov has been
   integrated into the core release with modifications to
   enable the new Ns_Gzip routine.  The module also provides
 

Re: [AOLSERVER] [ANN] AOLserver 4.5.0 released!

[Nathan Folkman]

The latest Tcl version (8.4.13) works great! Haven't spent any amount of 
time testing with upcoming Tcl 8.5, but it also should work fine I believe.


Not sure exactly what memory issues you are referring to, but if you are 
building Tcl multi-threaded, then you are using the threaded allocator 
(former Zippy allocator) which is simple x2 allocator. The allocator is 
optimized for front-end performance, and delivers pretty much 0% lock 
contention. This comes at the cost of wasted bytes through bucket size 
allocations, and also there is currently no garbage collection 
facilities provided.


We recently released a new version of the Tcl memory allocator code 
which does provide garbage collection, but it was unclear in our testing 
if that approach would ever really be valid. If high performance is your 
goal, then the Tcl threaded allocator is probably best for you. 
Alternately you could try something like Hoard, or even Google's tcmalloc.


Hope that helps!

- n

[EMAIL PROTECTED] wrote:

What version of Tcl is recommended for use with AOLserver 4.5.0?

Have memory usage/depletion issues been resolved in AOLserver 4.5.0, when
using the recommended version of Tcl?

Harvey Lunenfeld
Internet Corporation
[EMAIL PROTECTED]





- Original Message - From: "Dossy Shiobara" <[EMAIL PROTECTED]>
To: 
Sent: Tuesday, June 27, 2006 3:55 PM
Subject: [AOLSERVER] [ANN] AOLserver 4.5.0 released!



AOLserver 4.5.0 Released
June 27, 2006


   On behalf of the AOLserver Team, I have the honor of announcing the
   latest major release of AOLserver: 4.5.0.

   AOLserver 4.5.0 is a major upgrade including several new Tcl 
commands,

   C API's, and resource management improvements.

   A summary of changes is provided at the end of this document.


WHAT IS AOLSERVER?

   AOLserver is America Online's Open-Source web server.  AOLserver is
   the backbone of the largest and busiest production environments in
   the world.  AOLserver is a multithreaded, Tcl-enabled web server
   used for large scale, dynamic web sites.

   Visit the project's website:

   http://aolserver.com/


HOW CAN I GET AOLSERVER?

   Download the source code from SourceForge:

   http://aolserver.com/downloads/aolserver-4.5.0-src.tar.gz


I FOUND A BUG!  WHAT DO I DO?

   File it in the Bug Tracker at SourceForge:

   http://aolserver.com/sf/bugs


IS THERE A MAILING LIST?  WHERE'S THE USER COMMUNITY?

   Yes!  There's an announcements-only and a general discussion mailing
   list.  Instructions on how to subscribe are here:

   http://aolserver.com/lists.php

   There is also a wiki-web set up for AOLserver:

   http://aolserver.com/wiki/


CHANGES SINCE LAST RELEASE?

AOLserver 4.5 - June 2006
-

AOLserver 4.5 is a major upgrade including several new Tcl commands,
C API's, and resource management improvements.


Contents:


New and Updated Binaries
New Modules
New and Updated Tcl Commands
New and Updated C Routines
Additional Updates
Changes to the Build Process
Bug Fixes
Known Issues


New and Updated Binaries:


libnsd:
   The AOLserver library now includes an entry point suitable
   for loading into an ordinary, thread-enabled, tclsh, e.g.,:

   # tclsh
   % load /usr/local/aolserver/lib/libnsd.so
   % ns_time

   This provides the full set of Tcl commands within the
   AOLserver core although several commands will generate
   errors for lack of an HTTP request context (e.g., ns_conn).

nstclsh:
   A simple Tcl-shell linked against libnsd. It's equivalent
   to loading libnsd into an ordinary tclsh as described above.


New Modules:
---

nszlib:
   The popular nszlib module written by Vlad Seryakov has been
   integrated into the core release with modifications to
   enable the new Ns_Gzip routine.  The module also provides
   the "ns_zlib" command for compressing and uncompressing
   strings and files.

nsproxy:
   The nsproxy modules provides the "ns_proxy" command which
   enables sending Tcl scripts to a proxy process connected
   via a pipe.  The proxy includes core Tcl commands as well
   as AOLserver commands from the libnsd library. Evaluating
   scripts in a proxy process can be used to isolate and/or
   timeout thread-unsafe or otherwise unsafe 3rd party code.


New and Updated Tcl Commands:


The following Tcl commands are new or updated from the last release.
As few of the man pages have yet to be written, see the source code
for usage details and options.

ns_pools:
   The "ns_pools" command enables configuration of one or more
   pools of connection processing threads. The pools allow
   certain requests to be handled by specific threads. This
   could, for example, ensure multiple long running requests
   don't block other short running requests.  Pools are selected
   based on method/url pairs similar to the mappings managed
   by the "ns_register_proc" command.  By default

[AOLSERVER]

[Nathan Folkman]

If you haven't already, would you mind filing this as a bug against 4.5 
over at SourceForge? I set up a new category for 4.5 bugs. I'll try to 
take a look one of these evenings. Also, please provide a fail case. Thanks!


- n

[EMAIL PROTECTED] wrote:
Now...  if I visit a bogus url, I *still* receive the default 
aolserver not found page.  If I visit /404.adp directly, I 
successfully get the "sorry not found" message.  So, the redirects 
section seems to have no affect at server startup.  I've tried a few 
other methods with the same results.


I've gone back and verified that all of this works correctly under 
4.0.10.  I guess either this is a bug, or just a change in how to 
handle this.


Has anyone else run into this problem?  I'd appreciate if anybody 
could give a bit of time to sorting this out.


Yes, I reported this problem a few days ago, and should have mentioned 
that I was running the CVS head of 4.5, so it looks to me like it may 
be a genuine 4.5 bug.


-john


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to 
<[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the 
Subject: field of your email blank.



--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] Scaling at the high end

[Nathan Folkman]

In terms of distributed key/value pair data structures, there are two 
additional methods we've used here at AOL:


* AV (auto variables)
* NV (network variables)

AV's are a read-only data structure that under the covers is built on a 
sorted list for quick access, and small size. The files get built and 
then distributed to the front-end servers where the updated versions are 
automatically swapped in by code that looks at the last modification 
time for the AV file, and also some additional code which maintains 
reference counts for AV usage across interps. We tend to use RDIST to 
distribute these files from the server that creates them to the front-ends.


NV's are best suited for relatively small data sets which have frequent 
updates. The classic example is a stock ticker application. Updates are 
"streamed" to connected front-ends via persistent DCI_RPC connections 
between the NV writer and those front-end servers.


Both of the mechanisms are similar to how you would use NSV (NaviServer 
variables) locally, but can be used with a farm of connected front-end 
hosts. NV's also provide some additional functionality that you don't 
get with NSV's out the box, for example, a configuration option that 
will periodically dump the contents from memory to disk, as well as 
reading the version stored on disk at startup for persistent data 
storage across restarts.


A third approach we use is to use our SOB (small object blaster) servers 
and APIs to store data. The data can be anything, but most often is a 
Tcl list. The SOB APIs provide various caching levels, which depending 
on your application, might help with memory size since you'd only be 
caching data that was requested, as opposed to loading an entire data 
set into memory as you would with NSVs, AVs, and NVs.


Anyhow, these APIs are part of the nsdci AOLserver module which we will 
be releasing soon.


- n

[EMAIL PROTECTED] wrote:

On Monday 03 July 2006 08:33, John Buckman wrote:
  

One of the reasons I want to use Berkeley DB is that I'd like every
web page string to be a BDB database lookup, allowing wiki-style
correcting of strings on a web page (ie, anyone can correct, on the
spot, any translated text on any page).  SQL just wouldn't work with
that design, with a dozen or so db lookups per page.



So the design requires a number of queries, too many for responsive 
performance from an SQL database. However, if you use the BDB database, you 
will still have to go through some sort of database API, either the one which 
exists, or a custom designed API. The purpose of the built in API is to share 
connections across threads and even virtual servers, but it isn't really 
suited for simple key/value lookup. 

If the number of key/value pairs is small, everything could be loaded into 
memory, into an nsv_array. 

If the number of key/value pairs is large, then the solution is to load pairs 
into memory only on first use. Maybe a high-use cache could be developed and 
pre-loaded on server startup. If the cache will eventually grow too large, 
then ns_cache could be used to expire stale data. Maybe BDB doesn't have the 
option to unload memory mapped regions. If this is true, then ns_cache would 
be the better choice to control memory usage. 

But none of these solutions deals with how the key/value pairs are added to 
the database and distributed to one or more AOLservers. At some point there 
has to be a single master list of these pairs. Loading all these into memory, 
or into a BDB or DJB cdb requires only one SQL query. You can even do one 
query to write the database on the master, then distribute this with rsync or 
whatever to replace the old database.


Overall, it seems like BDB or cdb would have the same memory requirements of 
an nsv array, but they require some kind of API to get to this memory. Since 
it is just key/value pairs, this API overhead is a total waste: you have to 
use it even when the data is already in memory. 


tom jackson


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.
  



--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] Building 4.5 on OS X - .dylib vs. .a

[Nathan Folkman]

How are you building Tcl and AOLserver?

- n

[EMAIL PROTECTED] wrote:

Oh boy, another OS X build problem from me... :(

-
ranlib: can't open file: 
/Users/bas/dev/aolserver-4.5.0/lib/libnsthread.a (No such file or 
directory)

make[1]: *** [install-dll] Error 1
make: *** [install-bins] Error 1
geefive:~/dev/src/aolserver-4.5.0 bas$ ls 
/Users/bas/dev/aolserver-4.5.0/lib
libnsthread.dylib   libtcl8.4.dylib 
libtclstub8.4.a tcl8.4  tclConfig.sh

geefive:~/dev/src/aolserver-4.5.0 bas$
-

As you can see, this is a ".dylib" on OS X.

Anoyone seen this before and/or have a fix?

Cheers,
Bas.


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to 
<[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the 
Subject: field of your email blank.



--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Re: [AOLSERVER] Scaling at the high end (was Re: [AOLSERVER] build nsd w/o locking?)

[Nathan Folkman]

Speaking of SOB, we're working on getting the code to our "dci (Digital 
City) module" ready for release to the Open source community. This 
module contains the following items which folks may find interesting:


* SOB - Dossy described this below. Basically it's what we use at AOL to 
"forward cache" content.

* Network Poll - Provides a simple mechanism to create member facing polls.
* Network Tally - Allows one to aggregate counts for keys. We've used 
this in the past to track things like user views of a particular venue.
* Network Comment Boards - Allows one to easily create and add comment 
boards to your application.
* Network NV (network variables) - Similar to nsv, but allows updates to 
be distributed to many clients. Allows you to update variables on a 
single NV writer, and have those changes broadcast out to many front-end 
readers.
* Network Eval - Allows you to send Tcl scripts to be evaluated to many 
other servers.


Those are the really big things. They've been at use at AOL since the 
early Digital City days, and although relatively simplistic in their 
design - most assume single-writer for instance, they have shown 
themselves to be quite robust.


Stay tuned for the release of the "nsdci" module! ;-)

- n


[EMAIL PROTECTED] wrote:

On 2006.07.03, John Buckman <[EMAIL PROTECTED]> wrote:
  

In my experience, many  applications that use SQL actually only need
key-lookup capability [...]



Which is why, at AOL, a large part of the web publishing architecture
utilizes Small Object Broker (SOB), which can be set up in a distributed
(master-slave style) network based service with client-side read
caching with flush-on-write.  This lets you simply scale your key-based
query infrastructure horizontally.

Initially, I think some of the "conventional wisdom" at the
high-scalability end runs counter to what the majority of web developers
have discovered and learned because almost any reasonable solution will
work "in the small" which is where most web projects live.  In those
problem sets, tools and solutions which have a shallower learning curve
and offer more functionality "out of the box" yield faster development
times and empower developers to be productive with a smaller skill set.
However, there's a line (I couldn't yet tell you exactly where it is),
that once you cross it, the characteristics of the problem set changes
and the optimal (and sometimes only) solutions become very specialized.
It's at this end of the spectrum where AOLserver truly shines, but it's
a very small set.

John, it's fantastic to see you're actively looking at and working with
AOLserver.  From the list archives, it looks like your first message
came through around 15 Apr 2006.  I'm hoping that your interest in
AOLserver might mean that the upcoming version of Lyris may replace
tclhttpd with AOLserver?  Or are you working with AOLserver for
something different (i.e., Magnatune)?

-- Dossy

  



--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.