At 11:52 PM 4/15/00 +0000, [EMAIL PROTECTED] wrote:
>On Sun, Apr 16, 2000 at 09:28:56AM +0300, Stas Bekman wrote:
> > On Sat, 15 Apr 2000 [EMAIL PROTECTED] wrote:
> >
> > > > > I wrote a very small perl engine
> > > > > for phhttpd that worked within it's threaded paradigm that sucked
> up a
> > > > > neglibible amount of memory which used a very basic version of
> > > > > Apache's registry.
> > > >
> > > > Can you explain how this uses less memory than mod_perl doing the same
> > > > thing? Was it just that you were using fewer perl
> interpreters? If so, you
> > > > need to improve your use of apache with a multi-server setup. The
> only way
> > > > I could see phttpd really using less memory to do the same work is
> if you
> > > > somehow managed to get perl to share more of its internals in
> memory. Did
> > > > you?
> > >
> > > Yep very handily I might add ;-). Basically phhttpd is not process
> > > based, it's threaded based. Which means that everything is running
> > > inside of the same address space. Which means 100% sharing except for
> > > the present local stack of variables... which is very minimal. In
> > > terms of the perl thing... when you look at your processes and see all
> > > that non-shared memory, most of that is stack variables. Now most
> > > webservers are running on single processor machines, so they get no
> > > benefit from having 10s or even 100s of copies of these perl stack
> > > variables. Its much more efficient to have a single process handle
> > > all the perl requests. On a multiprocessor box that single process
> > > could have multiple threads in order to take advantage of the
> > > processors. See..., mod_perl stores the stack state of every script
> > > it runs in the apache process... for every script... copies of it,
> > > many many copies of it. This is not efficient. What would be
> > > efficient is to have as many threads/processes as you have processors
> > > for the mod_perl engine. In other words seperate the engine from the
> > > apache process so that there is never unneccesary stack variables
> > > being tracked.
> >
> > I'm not sure you are right by claiming that the best performance will be
> > achieved when you have a single process/thread per given processor. This
> > would be true *only* if the nature of your code would be CPU bound.
> > Unfortunately there are various IO operations and communications with
> > other components like RDBMS engines, which in turn have their IO as well.
> > Given that your CPU is idle while the IO operation is under process, you
> > could use the CPU for processing another request at this time.
> >
> > Hmm, that's the whole point of the multi-process OS. Unless I
> > misunderstood your suggestion, what you are offering is a kinda DOS-like
> > OS where there is only one process that occupies CPU at any given time.
> > (well, assuming that the rest of the OS essential processes are running
> > somewhere too in a multi-processes environment.)
>
>That is an excellent point Stas. One that I considered a while ago
>but sort of forgot about when I started this thread. Hrmm... that
>brings up much more complex issues. Yes, your right that is my
>assumption, and that's because that's the case I'm working under 90%
>of the time. It's a horrible assumption though for the "community at
>large". Hmm... well, you've stumped me... that's a very very clear
>problem with the design I had in mind. There are ways around this I
>can see in my brain, but they are far from eloquent. If something
>were blocking on a network read it would stop the WHOLE perl engine...
>TERRIBLE!!!!, not usefull at all for anyone that's going to be doing
>something like that. Well, there must be a way around it... if anyone
>has any ideas please shoot them my way... this is a paradox of the nth
>order. Actually it's a problem for mod_perl too..., but it's not
>nearly as large of a problem than for the design I had in mind.
>
>Congrats Stas... good thinking.
>Thanks,
>Shane.
>(DOSlike isn't fair though! :>. Though I see your point... efficiency
>was the key element to what I was thinking, but I had mostly
>considered the CPU bound case... the network bound case hadn't really
>entered my mind. The way around this is horribly yucky from a
>programatic point of view... e-gads!)
I guess that's why you would want to try different mixes of numbers of
servers on the back-end to see which gives you the greatest performance.
Jeffrey Baker also brings IO issues up in his ApacheDBI posts about why a
single pooled connection of DBI handles is not so hot in the real world
when compared against the single handle cached per apache process.
If you are CPU bound, then it may be just as well as to have a few servers
chugging away. And limit the number that can be forked off on the back-end.
If you are IO bound, then you would launch many more. However, the Apache
model of restarts and forking is not entirely shabby. That fact is, that if
you find some apache processes cannot fulfill the task, then another one
can always be created even if you have set a general upper limit of
mod_perl processes that are allowed to stick around. In other words, Apache
can be set up to adapt to the load placed on it. And that should be "OK".
If you want the ultimate in clean models, you may want to consider coding
in Java Servlets. It tends to be longer to write Java than Perl, but it's
much cleaner as all memory is shared and thread-pooling libraries do exist
to restrict 1-thread (or few threads) per CPU (or the request is blocked)
type of situation.
However, I would stress that speed/threading is not the only name in the
game. Reliability is another concern. Frankly, I have seen a lot of buggy
servlets crash the entire servlet engine making a web site useless.
Generally if there is a core dump in an Apache/Mod_perl process, the worst
is that one request got hosed.
I am resigned to the fact that all languages are buggy, and so I like
engineering architectures that support buggy languages (as well as buggy
app code).
Later,
Gunther
__________________________________________________
Gunther Birznieks ([EMAIL PROTECTED])
Extropia - The Web Technology Company
http://www.extropia.com/
