Re: load balancing on apache

[ed phillips]

Jeff Beard wrote:
> 
> On Fri, 14 Dec 2001, Perrin Harkins wrote:
> 
> > > I _really_ hate so-called dedicated boxes. They're closed, nasty,
> > > inflexible and often don't work in _your_ situation. Doing smart
> > > session-based redirection can be hard with these boxes.
> >
> > You can make it work with homegrown solutions, but I've found the dedicated
> > load-balancing tools (at least Big/IP) to be effective and fairly easy to
> > work with, even with large loads, failover requirements, and more exotic
> > stuff like sticky sessions.  This is one area where the problem seems to be
> > well enough defined for most people to use an off-the-shelf solution.
> > They're often more expensive than they should be, but if you don't have
> > someone on hand who knows the ipchains or LVS stuff it can save you some
> > time and trouble.
> 
> I couldn't agree more. In terms of managability and scalability,
> the various software solutions simply add complexity to something that is
> already so. I've got some experience with Alteon AceDirectors and even though
> they seem little flakey at times, you do end up with true load balacing. (We
> have Cisco's solution deployed and they periodically have issues too.)
> 
> DNS round-robin should be avoided at all costs. It's half-assed at best. In
> the case of a failure those clients that have that IP cached are SOL.
> 
> On some of the systems that I've deployed we have a frontend proxy on the same box
> as the mod_perl with the mod_perl server listening on 127.0.0.1. This is
> behind an Alteon (or 2). You can put the proxy on a separate box as well but
> (I've seen some odd problems with TCP connections not working in this situation
> which I never fully understood but may have had to do with the Alteon being flakey.)
> 
> Anyway, my advice is to go with a hardware load balancer/intelligent IP switch.
> In the long term, it will pay for itself in the time recovered from *not* being
> spent on troubleshooting complex problems.
> 

yes. It's a money vs. time/knowledge thing. Plus the state of the free
software available. Anyone care to compare the features and power of
some of the opensource projects vs. the Big/IP's? Which are the more
promising opensource projects in this area?

It would be nice to use an open source solution, or at least be able to
offer it as an option, and I'd like to track the progress of some of the
more promising projects.

Ed

Ed

Re: Defeating mod_perl Persistence

[ed phillips]

Ged Haywood wrote:
> 
> Hi there,
> 
> On Tue, 11 Dec 2001, Jonathan M. Hollin wrote:
> 
> > When using Mail::Sender only the first email is sent on my mod_perl server.
> > When I investigated, I realised that the socket to the SMTP server was
> > staying open after the completion of that first email (presumably mod_perl
> > is responsible for this persistence).
> >
> > Is there any way to defeat the persistence on the socket while running my
> > script under mod_perl, or do such scripts always need to be mod_cgi?
> 
> The idea is for the mod_perl process to complete its job and get on
> with another as quickly as possible.  Waiting around for nameserver
> timeouts and such doesn't help things.
> 
> You might be better off re-thinking the design for use under mod_perl.
> This is a well-trodden path, have a browse through the archives.
> 

Yes, this has come up before. Ideally you want to separate out your mail
service and pass your mails to a queue. Then, wholly independent of your
app, your smtp server can negotiate with remote hosts and generally do
its thing. That is, you shouldn't even make your app wait for your SMTP
server to send an email before you free it to handle the next request.
This is analagous to using a proxy server to handle slowish clients. See
the guide, archives.


Ed

Re: Shutdown

[ed phillips]

"Khachaturov, Vassilii" wrote:
> 
> Hi Matt, do you have any news on planned re-appearance of
> current axkit docs and mailing list archives online? The net people
> memory is short, if you don't come back noone will know what AxKit is :-(
> Or did you abandon it in favor of smth better?
> 

Just to add to this. I've, several times, recommended Axkit to people,
only to have them not be able reach axkit.org or axkit.com and have them
no longer entertain the possibility of working with it

Re: [OT] Off Topic?

[ed phillips]

Robin Berjon wrote:
> 
> On Friday 12 October 2001 00:52, Ged Haywood wrote:
> > See attached.
> 
> Shouldn't that document go on perl.apache.org ? Last time you brought it up
> you thought that it wasn't perfect enough yet, but however relatively
> "imperfect" you may consider it to be (I think it's very good as is) it's
> certainly already helpful ! So can't someone simply commit it to the site ?
> Pretty please ?
> 

Yes, I read it and it looks good. Stas did not wait for the guide to be
perfect before he put it up. ;-) Can you remember what the guide was
like a couple of years ago? Put it up.

Ed

[OT]Re: The DEFINITIVE answer to: How much should I charge?

[ed phillips]

Tom Mornini wrote:
> 
> This whole thread can be answered very easily:
> 
> ANSWER: As much as you can.
> 
> That's it! That's the entire answer. Nothing else should figure in
> unless you
> personally wish to make exceptions for any reason you see fit.
> 
> Did the people who ask this question grow up and become educated in a
> part of the
> world where free markets and capitalism did not exist?
> 
> Perhaps in socialistic colleges in the U.S.? :-)
> 

If you mean when you say, "sociailistic colleges", very well funded
universities full of tenured radicals, then I'm guilty. ;-)

Those were my favorite professors!  But, I was never deluded into
thinking they had in any sense escaped the money economy. The star
tenured radicals such as Fred Jameson for example make as much or more
than a very well paid software developer so one must appreciate the
ironies.

As a freelancer you charge what the market will bear. Besides the extra
cost of benefits and the added tax liabilities one must also factor in
the assumption of risk if you want to come to a justification for
charging 100+ per hour. If you don't feel the need to justify, then you
merely say, "I charge the market rate." No tenured radical would
begrudge you. They know on what side their bread is buttered. ;-)

ed

Re: [OT] Re: What hourly rate to charge for programming?

[ed phillips]

Perrin Harkins wrote:
> 
> > Now take the amount you want to make and divide it by the number
> > of hours you came up with above ($40,000 / 1,000).  You get $40.
> > That's your target hourly rate.  And despite what they high-flying
> > .com weanies were saying a year ago, that's going to be a nice
> > living for a young guy unless you're smack in the middle of a
> > high-cost area and can't bother to cook your own meals.
> 
> Don't forget that self-employed people in the US must pay considerably more
> in social security, as well as covering the full cost of their own health
> insurance and other needed benefits.  $40K as a consultant is much less
> spendable money than $40K as an employee.
> - Perrin


Yes, that's an additional 7.5% for social security. In addition, you
have to take care of your own benefits, etc.

Market downturns can be a better time for contract work over full-time,
especially since stock options don't mean what they do during an upturn.
;-) And many employers don't have the resources to take on full-time
staff.

I'd recommend that you start to inch up your rate with new clients, and
that you try and see what your market will bear. Your target should be
$100+ in the U.S. for basic consulting and more for mod_perl specific
work, again if your market will bear it.

Good Luck,

Ed

Re: [ANNOUNCE] TicketMaster.com sponsors mod_perl development

[ed phillips]

"Aaron E. Ross" wrote:
> 
> On Fri, Sep 21, 2001 at 02:01:31AM +0800, Gunther Birznieks wrote:
> > "You can reach your goals.
> >
> > I'm living proof.
> >
> > beefcake.
> >
> > BEEFCAKE!!"
> >
> > -- Eric Cartman
> 
>  LOL!  sounds like a great project stas! thanks ticketmaster!


Yeah. Kudos to Ticketmaster for supporting a great Open Source project.

Re: [ANNOUNCE] TicketMaster.com sponsors mod_perl development

[ed phillips]

Congratulations to Stas, mod_perl, and the guide.

Excellent!

Ed


Stas Bekman wrote:
> 
> If you remember back in the end of April, I've posted to the list an
> unusual job seek request [1], where I was saying that I want some
> company to sponsor me to work full time on mod_perl 2.0 development.
> 
> Believe it or not my unusual request has been answered by Craig McLane
> from TicketMaster.com (which owns citysearch.com).
> 
> citysearch.com is a heavy user of mod_perl technology and interested in
> making sure that mod_perl technology get more and more mature and ensure
> their business' success.
> 
> So starting from this September I'm working on mod_perl 2.0
> development, a new documentation project (which you are welcome to
> join) and doing mod_perl advocacy through teaching at the conferences
> and other ways.
> 
> Currently the contract is for one year. But if everything goes well,
> and mod_perl 2.0 rocks the world even better than 1.x did we will see
> more support and sponsoring from TicketMaster.
> 
> This email's purpose:
> 
> - is to set a precedent for other business to sponsor mod_perl and
>related technologies. There are at least a few excellent developers
>that I know will jump on the opportunity of being able to do what
>they love full time.
> 
> - is to set a precedent for other developers to seek what they really
>want and read less stories about hi-tech recession, since good
>developers are always in demand. Therefore I hope that this email
>will encourage you to do that.
> 
> Notes:
> 
>[1] http://forum.swarthmore.edu/epigone/modperl/runvesay
> 
> _
> Stas Bekman  JAm_pH --   Just Another mod_perl Hacker
> http://stason.org/   mod_perl Guide  http://perl.apache.org/guide
> mailto:[EMAIL PROTECTED]   http://apachetoday.com http://eXtropia.com/
> http://singlesheaven.com http://perl.apache.org http://perlmonth.com/

Re: mod_proxy and mod_perl in guide

[ed phillips]

Thanks Vivek,

Andrei, use the front end to directly handle any binaries, static files,
etc.

I doubt they are generating of these on the fly.



Vivek Khera wrote:
> 
> > "AAV" == Andrei A Voropaev <[EMAIL PROTECTED]> writes:
> 
> AAV> In our system we have to pass large PDF files thru mod_perl to
> AAV> proxy and we noticed that it takes the same time as sending it
> AAV> directly to customer.
> 
> Why do you have to pass the PDF thru mod_perl?  Are you generating it
> on the fly?  If not, configure your proxy front end to intercept
> static documents like .pdf .txt .html etc. to be handled by the front
> end directly.  I use mod_rewrite for this, and my configs have been
> posted to this list at least twice.
> 
> --
> =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
> Vivek Khera, Ph.D.Khera Communications, Inc.
> Internet: [EMAIL PROTECTED]   Rockville, MD   +1-240-453-8497
> AIM: vivekkhera Y!: vivek_khera   http://www.khera.org/~vivek/

Re: modperl/ASP and MVC design pattern

[ed phillips]

[EMAIL PROTECTED] wrote:

> > Francesco, I believe that Ian was joking, hence the yikes before the name,
> > so  the above post is the documentation!
> >
> > Ed
> >
>
> .. so the best environment for the MVC++ design pattern is parrot/mod_parrot :)
> http://www.oreilly.com/news/parrotstory_0401.html
>
> Thanks
> Francesco
>

Exactly!

Wasn't Ian the one responsible for the mod_parrot MVC++ API?

ed

Re: modperl/ASP and MVC design pattern

[ed phillips]

Francesco Pasqualini wrote:

> - Original Message -
> From: <[EMAIL PROTECTED]>
> To: "Francesco Pasqualini" <[EMAIL PROTECTED]>
> Cc: <[EMAIL PROTECTED]>
> Sent: Friday, April 20, 2001 8:11 PM
> Subject: Re: modperl/ASP and MVC design pattern
>
> >
> > You can (I have) accomplish this with mod_perl and HTML::Mason, Mason's
> > root level autohandler can play the role of the JSP model 2
> > "controller servlet": dispatching logic processing to Perl objects (er,
> > beans) and "forwarding" to a view (with the Mason $m->call_next or
> > $m->comp mechanisms).  I think the Apache::Dispatch stuff can also
> > perform this role (haven't played with it to say for certain).  I'll
> > qualify this by saying MVC is not a end in itself, there are a lot of
> > modern requirements for flexible branding, client form factor appropriate
> > and locale specific presentations that require the view/controller part to
> > be a lot smarter than the traditional concepts of MVC that I've seen call
> > for.  I've been referring to these needs in my own engineering discussions
> > as (yikes) MVC++  :)
>
> ... this is really interesting, can you point me to documentation about
> "MVC++"
> thanks
> Francesco

Francesco, I believe that Ian was joking, hence the yikes before the name, so
the above post is the documentation!

Ed

Re: Can AxKit be used as a Template Engine?

[ed phillips]

Michael Alan Dorman wrote:

> Matt Sergeant <[EMAIL PROTECTED]> writes:
> > It depends a *lot* on the type of content on your site. The above
> > www.dorado.com is brochureware, so it's not likely to need to be
> > re-styled for lighter browsers, or WebTV, or WAP, or... etc. So your
> > content (I'm guessing) is pure HTML, with Mason used as a fancy way
> > to do SSI, with Mason components for the title bars/menus, and so
> > on. (feel free to correct me if I'm wrong).
>
> It is more sophisticated than that, but you're basically right.  I do
> pull some tagset-like tricks for individual pages, so it's not totally
> pure HTML, but yeah, if we wanted to do WebTV we'd be fscked.
>
> > AxKit is just as capable of doing that sort of thing, but where it
> > really shines is to provide the same content in different ways,
> > because you can turn the XML based content into HTML, or WebTV HTML,
> > or WML, or PDF, etc.
>
> Ah---well a web site that does all of that isn't what first comes to
> mind when someone talkes about doing a "static site"---though now that
> you've explained further, I believe I understand exactly what you
> intended.
>
> > I talk about how the current Perl templating solutions (including
> > Mason) aren't suited to this kind of re-styling in my AxKit talk,
> > which I'm giving at the Perl conference, so go there and come see
> > the talk :-)
>
> Heh.  I agree entirely with this assesment---I can conceptualize a way
> to do it in Mason, but the processing overhead would be unfortunate,
> the amount of handwaving involved would be enormous, and it would
> probably be rather fragile.
>
> > So I take back that people wouldn't be using Mason for static
> > content. I was just trying to find a simple way to classify these
> > tools, and to some people (I'd say most people), Mason is more on
> > the dynamic content side of things, and AxKit is more on the static
> > content side of things, but both tools can be used for both types of
> > content.
> >
> > (I hate getting into these things - I wish I'd never brought up
> > Mason or EmbPerl)
>
> Well I will say that you made an excellent point that hadn't really
> occured to me---I use XML + XSL for a lot of stuff (the DTD I use for
> my resume is a deeply reworked version of one I believe you had posted
> at one time), but not web sites, in part because I'm not currently
> obligated to worry about "other devices"---so I don't exactly regret
> getting you to clarify things.
>
> Could I suggest that a better tagline would be that AxKit is superior
> when creating easily (re-)targetable sites with mostly static content?
> It might stave off more ignorant comments.
>
> Mike.

Matt,

I've also found your use of "static" to describe "transformable" or
"re-targetable"(unfortunate
word)" content to be confusing. This discussion helps clarify things, a
little. ;-)

Ed

Re: Fast DB access

[ed phillips]

Matthew Kennedy wrote:

> I'm on several postgresql mailing lists and couldn't find a recent post
> from you complaining about 6.5.3 performance problems (not even by an
> archive search). Your benchmark is worthless until you try postgresql
> 7.1. There have been two major releases of postgresql since 6.5.x (ie.
> 7.0 and 7.1) and several minor ones over a total of 2-3 years. It's no
> secret that they have tremendous performance improvements over 6.5.x. So
> why did you benchmark 6.5.x?
>
> This is a good comparison of MySQL and PostgreSQL 7.0:
>
> "Open Source Databases: As The Tables Turn" --
> http://www.phpbuilder.com/columns/tim20001112.php3
>
> > We haven't tried this one. We are doing a project on mysql. Our preliminary 
>assessment is, it's a shocker. They justify not having commit and rollback!! Makes us 
>think whether they are even lower end than MS-Access.
>
> Again, checkout PostgreSQL 7.1 -- I believe "commit" and "rollback" (as
> you put it) are available. BTW, I would like to see that comment about
> MS-Access posted to pgsql-general... I dare ya. :P
>
> Matthew

You can scale any of these databases; Oracle, MySQL or PostgreSQL, but please research 
each one thoroughly and tune it properly before you do your benchmarking.  And, again, 
MySQL does support transactions now. Such chutzpah for them to have promoted an "atomic
operations" paradigm for so long without supporting transactions! But that discussion 
is moot now.

Please be advised that MySQL is threaded and must be tuned properly to handle many 
concurrent users on Linux. See the docs at http://www.mysql.com  The author of the PHP 
Builder column did not do his research, so his results for MySQL on Linux are way off.
Happily, though, even he got some decent results from PostgreSQL 7.0.

The kernel of wisdom here:  If you are going to use one of the Open Source databases, 
please use
the latest stable release (they improve quickly!) and please either hire someone with 
some expertise installing and administering, and tuning your database of choice on 
your platform of choice or do the research thoroughly yourself.

Ed

Re: Varaible scope & memory under mod_perl

[ed phillips]

agh!

check the headers!


Steven Zhu wrote:

> How could I unsubscribe from [EMAIL PROTECTED] you so
> much.Steven.
>
>  -Original Message-
>

Re: Not even beginning - INSTALL HELP

[ed phillips]

If you are going to upgrade gcc for RH 7.0, I reccomend the
new source RPM for gcc to be found in the updates directory
on any redhat mirror site.  In fact, if you are sticking with RH
you should see about updating a number of things.

23,

Ed

"G.W. Haywood" wrote:

> Hi there,
>
> On Tue, 27 Feb 2001, A. Santillan Iturres wrote:
>
> > I have Apache 1.3.12 running on a RedHat 7.0 box with perl, v5.6.0 built for
> > i386-linux
> > I went to install mod_perl-1.25:
> > When I did:
> > perl Makefile.PL
> > I've got a:
> > Segmentation fault (core dumped)
>
> Did you build your Perl yourself?  Sounds like there's a problem with
> it.  Check out the mod_perl List archives for problems with gcc (the C
> compiler) that was shipped with RedHat 7.0.  You should probably get
> that replaced to start with.  (Or use Slackware - sorry:)
>
> 73,
> Ged.

Re: is morning bug still relevant?

[ed phillips]

Please use the  MySQL modules list. Responses are timely.
;-)

ed

Subscribe: 




Vivek Khera wrote:

> > "SV" == Steven Vetzal <[EMAIL PROTECTED]> writes:
>
> SV> Greetings,
> >> to say "ping doesn't work in all cases" without qualifiying why and/or
> >> which drivers that applies to.
>
> SV> We've had to write our own ->ping method for the MySQL DBD
> SV> driver. Our developer tried to track down a maintainer for the
> SV> DBD::msql/mysql module to submit a diff, but to no avail.
>
> How old a version are you talking about?  In any case, according to
> CPAN, the DBD::mysql module is "owned" by
>
> Module id = DBD::mysql
> DESCRIPTION  Mysql Driver for DBI
> CPAN_USERID  JWIED (Jochen Wiedmann <[EMAIL PROTECTED]>)
> CPAN_VERSION 2.0414
> CPAN_FILEJ/JW/JWIED/Msql-Mysql-modules-1.2215.tar.gz
> DSLI_STATUS  RmcO (released,mailing-list,C,object-oriented)
> INST_FILE(not installed)
>
> and I *know* he's responsive to that email address at least as of a
> month or so ago, as we exchanged correspondence on another matter.
>
> --
> =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
> Vivek Khera, Ph.D.Khera Communications, Inc.
> Internet: [EMAIL PROTECTED]   Rockville, MD   +1-240-453-8497
> AIM: vivekkhera Y!: vivek_khera   http://www.khera.org/~vivek/

[OT]Re: mod_perl advocacy project resurrection

[ed phillips]

Aristotle from the Ars Rhetorica on money:

Money will not make you wise, but it will bring a wise man to your door.


Robin Berjon wrote:

> At 12:39 06/12/2000 -0800, brian moseley wrote:
> >> ActiveState has built an Perl/Python IDE out of Mozilla:
> >>  http://www.activestate.com/Products/Komodo/index.html
> >
> >too bad it's windows only :/
>
> That's bound to change. I think AS will release it on all platforms where
> Moz/Perl/Python run when it's finished. The current release is very
> unstable anyway.
>
> -- robin b.
> All paid jobs absorb and degrade the mind. -- Aristotle
>
> -
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Apache::Registry() and strict

[ed phillips]

Ron,

This is a greivous FAQ.  Please read the guide at
http://perl.apache.org/guide

You'll find much more than this question answered.

Ed



Ron Rademaker wrote:

> Hello,
>
> I'm just starting with mod_perl and I'm using Apache::Registry(). The
> second line after #!/usr/bin/perl -w is use strict;
> But somehow variables I use in the script are still defined if I execute
> the script again, in one of the script I said undef $foo at the
> end, but I don't think this is the way it should be done, but it did work.
> Anyone knows what could be causing this??
>
> Ron Rademaker
>
> PS. Please CC to me because I'm not subscribed to this mailinglist

Re: Apache trouble reading in large cookie contents

[ed phillips]

Explictly echoing Gunther, don't go there!

Use cookies, think crumbs of info, as flyweights.  Significant chunks of data need
to be passed and stored
in other ways.

Ed

Gunther Birznieks wrote:

> Caveat: even if you modify apache to do larger cookies, it's possible that
> there will be a set of browsers that won't support it.
>
> At 04:48 PM 10/20/00 -0700, ___cliff rayman___ wrote:
> >i'm not an expert with this, but, a quick grep for your error in
> >the apache source (mine is still 1.3.9 ) and some digging yield:
> >
> >./include/httpd.h:#define DEFAULT_LIMIT_REQUEST_FIELDSIZE 8190
> >
> >so you're right, 8K is currently the apache limit. if you try to change
> >this value in
> >the source code, you will probably also have to muck with IOBUFSIZE and
> >possibly other things as well.  IOBUFSIZE is 8192 and the
> >DEFAULT_LIMIT_REQUEST_FIELDSIZE is set to 2 bytes below that to make
> >room for the extra \r\n after the last header.
> >
> >looks like you'll have to take responsibility for mucking with the apache
> >source, or
> >sending smaller cookies and using some other techniques such as HIDDEN fields.
> >
> >
> >--
> >___cliff [EMAIL PROTECTED]http://www.genwax.com/
> >
> >"Biggs, Jody" wrote:
> >
> > > I'm having trouble when a browser sends a fair sized amount of data to
> > > Apache as cookies - say around 8k.
> > >
> >
> > > Apache then complains (and fails the request) with
> > > a message of the sort:
> >
> > > [date]  [error] [client 1.2.3.4] request failed: error reading the headers
> >
> > > I assume this is due to a compile time directive to Apache specifying the
> > > maximum size of a header line.
> > >
>
> __
> Gunther Birznieks ([EMAIL PROTECTED])
> eXtropia - The Web Technology Company
> http://www.extropia.com/

Re: Forking in mod_perl?

[ed phillips]

I hope it is clear that you don't want fork the whole server!

Mod_cgi goes to great pains to effectively fork a subprocess, and
was the major impetus I believe for the development of
the C subprocess API. It  (the source code for
mod_cgi) is a great place to learn some of the
subtleties as the Eagle book points out. As the Eagle book
says, Apache is a complex beast. Mod_perl gives
you the power to use the beast to your best advantage.

Now you are faced with a trade off.  Is it more expensive to
detach a subprocess, or use the child cleanup phase to do
some extra processing? I'd have to know more specifics to answer
that with any modicum of confidence.

Cheers,

Ed


"David E. Wheeler" wrote:

> ed phillips wrote:
> >
> > Hi David,
> >
> > Check out the guide at
> >
> > http://perl.apache.org/guide/performance.html#Forking_and_Executing_Subprocess
> >
> > The Eagle book also covers the C API subprocess details on page 622-631.
> >
> > Let us know if the guide is unclear to you, so we can improve it.
>
> Yeah, it's a bit unclear. If I understand correctly, it's suggesting
> that I do a system() call and have the perl script called detach itself
> from Apache, yes? I'm not too sure I like this approach. I was hoping
> for something a little more integrated. And how much overhead are we
> talking about getting taken up by this approach?
>
> Using the cleanup phase, as Geoffey Young suggests, might be a bit
> nicer, but I'll have to look into how much time my processing will
> likely take, hogging up an apache fork while it finishes.
>
> Either way, I'll have to think about various ways to handle this stuff,
> since I'm writing it into a regular Perl module that will then be called
> from mod_perl...
>
> Thanks,
>
> David

Re: Forking in mod_perl?

[ed phillips]

Hi David,

Check out the guide at

http://perl.apache.org/guide/performance.html#Forking_and_Executing_Subprocess

The Eagle book also covers the C API subprocess details on page 622-631.

Let us know if the guide is unclear to you, so we can improve it.

Ed


"David E. Wheeler" wrote:

> Hi All,
>
> Quick question - can I fork off a process in mod_perl? I've got a piece
> of code that needs to do a lot of processing that's unrelated to what
> shows up in the browser. So I'd like to be able to fork the processing
> off and return data to the browser, letting the forked process handle
> the extra processing at its leisure. Is this doable? Is forking a good
> idea in a mod_perl environment? Might there be another way to do it?
>
> TIA for the help!
>
> David
>
> --
> David E. Wheeler
> Software Engineer
> Salon Internet ICQ:   15726394
> [EMAIL PROTECTED]   AIM:   dwTheory

Re: open(FH,'|qmail-inject') fails

[ed phillips]

Greg Stark wrote:

> A better plan for such systems is to have a queue in your database for
> parameters for e-mails to send. Insert a record in the database and let your
> web server continue processing.
>
> Have a separate process possibly on a separate machine or possibly on multiple
> machines do selects from that queue and deliver mail. I think the fastest way
> is over a single SMTP connection to the mail relay rather than forking a
> process to inject the mail.
>
> This keeps the very variable -- even on your own systems -- mail latency
> completely out of the critical path for web server requests. Which is really
> the key measure that dictates the requests/s you can serve.
>

Exactly, Greg.  This is homologous to proxy serving http requests. Ideally, the
data/text
should be relayed to a separate, dedicated mail server.  This has come up
repeatedly
for me on performance tuning projects. If there are a number of mail processes
negotiating
with remote hosts even running on the same machine as you are web serving from,
you may,
under significant load, degrade performance.

Re: OT: Server-push client page reload

[ed phillips]

A very impressive 95 lines o' Perl Randal!

Ed



"Randal L. Schwartz" wrote:

> > "Michael" == Michael Nachbaur <[EMAIL PROTECTED]> writes:
>
> Michael> This is off-topic, but I need an answer pretty quick, and I
> Michael> *am* writing this app using mod_perl, so its sorta related
> Michael> (also, I don't want the headache of re-subscribing to a new
> Michael> list).
>
> Michael> You know those online web-based tech support chat systems?
> Michael> Its commonly frame based, but its just like IRC, but over
> Michael> HTML.  when a user posts a message it immediatly pops up on
> Michael> the chat frame, and you submit your message through a
> Michael> regular-ol' HTML form.  I don't think this is an applet,
> Michael> because this works in all sorts of browsers.  I think its
> Michael> javascript, but I'm not sure. My main question, is when the
> Michael> server knows that a new message has been posted, how does it
> Michael> push that new page out to the client web browser?  I'm used
> Michael> to all page-views originating from the client...not the
> Michael> server.
>
> Michael> Any ideas?
>
> Server push is not universal.  Client Pull is more available.
> See  for an example
> of a very short client-pull webchat, from an upcoming WebTechniques
> column (past columns at ).
>
> --
> Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
> <[EMAIL PROTECTED]> http://www.stonehenge.com/merlyn/>
> Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
> See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!

Re: [OT] [JOB] mod_perl and Apache developers wanted

[Ed Phillips]

It is interesting and and somewhat ironic that the Engineering
dep at eToys is part of the open source community and culture
while their management's behavior was so disastrously misguided
and so misunderstanding of net culture and precedent.
They shot themselves in the foot pretty badly.

Would eToys have paid for the legal expenses of the Etoy group
if they weren't clued in by their Engineering department? Have
they learned a hard lesson?

Perrin is an exemplary figure, and I commiserate with him, but
some basic precedents of net culture need to be respected for the
network to function and the culture to flourish. If we had not
protested the attempted eToys domain grab, and I was one
who protested, they may have never recanted and  Etoy might
still be fighting at absurd personal cost.

Cheers,

Ed




Paul Singh wrote:

> Regardless of what eToys' intentions were, the way I see it, this was a case
> in which a billion dollar corporation (well, at least it was back then)
> filed suit against a handful of artists who had the etoy.com domain way
> before eToys came along.  eToys had no legitimate stake to the domain... and
> I don't associate legitimacy with the law... they seldom coincide.  So if
> this isn't a case of the bigger guy bullying the little guy, what is it?
> Granted, I have a distant association with the eToy crew so my opinions will
> be biased... however, even with staying to the facts and ignoring eToys'
> motivations, their actions alone reek of unfairness (at best).
>
> Of course, this says little of what type of work environment eToys is and
> the people that work there... but it does comment on the corporation and the
> people running it.
>
> But as you said, this is definitely off-topic, and I will cease further
> comment... take care.
>
> - jps
>
> > -Original Message-
> > From: Perrin Harkins [mailto:[EMAIL PROTECTED]]
> > Sent: Friday, June 16, 2000 4:48 PM
> > To: Paul Singh
> > Cc: ModPerl Mailing List
> > Subject: RE: [OT] [JOB] mod_perl and Apache developers wanted
> >
> >
> > On Thu, 15 Jun 2000, Paul Singh wrote:
> > > While that may be true (as with many publications), I hope you're not
> > > denying the facts of this case
> >
> > The basic facts are correct: eToys received complaints from parents about
> > the content their children found on the etoy.com site and, after failing
> > to reach an agreement with the site's operators, filed a lawsuit involving
> > trademarks which led to etoy being ordered to shut down their site by a
> > judge.
> >
> > Slashdot's coverage ignored or underreported some aspects of the situation
> > (the motivation behind the lawsuit, epxloitation of the name confusion on
> > the part of etoy), and reported some conjecture and pure flights of fancy
> > as fact (evil intentions, scheming lawyers).  You have no idea how painful
> > it is to read things like that from a source that you trust and consider
> > part of your community.  I guess I should have known better though:
> > Slashdot is an op/ed site.  If you want the news, you still have to read
> > the New York Times (who had much more accurate coverage of the events).
> >
> > Anyway, I don't claim that eToys was right to take legal action, just that
> > the reports about an evil empire were greatly exaggerated and that eToys
> > is a good place to work, full of good people.  Anyone who doesn't believe
> > me at this point probably never will, so I'm going to stop spamming the
> > list about this subject and go back to spamming about mod_perl.
> >
> > - Perrin
> >

Re: [OT] Contract Language for free software

[Ed Phillips]

Please excuse the horrible formatting,
The version below should be more readable.


Ed Phillips wrote:

> Hi All,
>
> This is very OT, but is related to mod_perl.  I have a contract as yet
> unsigned
> with a Web Company to possibly rewrite their horrible perl 4 era CGI apps
> as nice  clean mostly OO mod_perl apps. It's a simple job, rather generic.
>
> I usually license my code with the Artistic License, so this is not a
> Licensing
> question, exactly. In the contract, they have sent me, they have a
> paragraph
> that signs the rights to all inventions over to them.
>
> From my perspective, what little invention there might be in these apps,
> is drawn directly from freely available documentation and advice, and
> the apps will of course rely heavily on CPAN modules, as well as
> things I've written before. So, what I'm looking
> for is advice or a paragraph that establishes that the greater part of
> invention
> is already public, that these programs use free software, and that
> non-specific
> to their business inventions, if any should be returned to the community,
> etc.
>
> I'm not a lawyer. :-) In the past, I've just ignored such paragraphs and
> I've
> lowered the boom in a banner at the top of my code.  But, I'd like to put
> something
> in any contract that I sign and these people have no idea how to deal with
> free software.
> They want me to sign an NDA now that I've seen their hideous spaghetti,
> which is
> of course full of copyright notices.  I told them that if anyone stole
> this code, they would
> need their head checked!
>
> If someone has crafted any pro free software contracts that cover
> invention and
> they would like to share, I'd be grateful.
>
> Cheers,
>
> Ed

[OT] Contract Language for free software

[Ed Phillips]

Hi All,

This is very OT, but is related to mod_perl.  I have a contract as yet
unsigned
with a Web Company to possibly rewrite their horrible perl 4 era CGI
apps
as nice  clean mostly OO mod_perl apps. It's a simple job, rather
generic.

I usually license my code with the Artistic License, so this is not a
Licensing
question, exactly. In the contract, they have sent me, they have a
paragraph
that signs the rights to all inventions over to them.

>From my perspective, what little invention there might be in these apps,

is drawn directly from freely available documentation and advice, and
the apps will of course rely heavily on CPAN modules, as well as
things I've written before. So, what I'm looking
for is advice or a paragraph that establishes that the greater part of
invention
is already public, that these programs use free software, and that
non-specific
to their business inventions, if any should be returned to the
community, etc.

I'm not a lawyer. :-) In the past, I've just ignored such paragraphs and
I've
lowered the boom in a banner at the top of my code.  But, I'd like to
put something
in any contract that I sign and these people have no idea how to deal
with free software.
They want me to sign an NDA now that I've seen their hideous spaghetti,
which is
of course full of copyright notices.  I told them that if anyone stole
this code, they would
need their head checked!

If someone has crafted any pro free software contracts that cover
invention and
they would like to share, I'd be grateful.

Cheers,

Ed

Re: [OT now] Re: Template techniques

[Ed Phillips]

I'm just using XML on the backend for content management and as
a way to standardize what I recieve from partners and content folks,
then storing parsed content in a database from which I output text, HTML,
and/or XML.

XML::Parser suits quite fine for the above. So, Perl has plenty of XML support,
imo.

I've taken a look at what Matt is up to and I'm intrigued, but don't have a need
for it as yet.

Joshua,  what is the itch that you are scratching if you care to opine?

Ed


Drew Taylor wrote:

> Joshua Chamas wrote:
> >
> > Perrin Harkins wrote:
> > >
> > > On Fri, 9 Jun 2000, Drew Taylor wrote:
> > > > I really like the fact that templates can be compiled to perl code &
> > > > cached. Any others besides Mason & EmbPerl (and TT in the near future)?
> > >
> > > Sure: Apache::ePerl, Apache::ASP, Text::Template, and about a million
> > > unreleased modules that people wrote for their own use.  (I think writing
> > > a module that does this should be a rite of passage in Perl hacking.)
> > >
> >
> > For my second rite of passage, I'm hacking XML::XSLT
> > integration into Apache::ASP for realtime XSLT document
> > rendering with a sophisticated caching engine utilizing
> > Tie::Cache.  Moving forward, the XML buzzword seems to be
> > just about a necessity.
> >
> > Take it as a sign of respect Matt :)
> Cool! The thing that perl is missing the most right now is XML support.
> The more (and the sooner :-) packages support XML easily and natively,
> the better. I'm still an XML newbie, so all this recent perl XML
> development is very exciting for me!
>
> --
> Drew Taylor
> Vialogix Communications, Inc.
> 501 N. College Street
> Charlotte, NC 28202
> 704 370 0550
> http://www.vialogix.com/

Re: [benchmark] DBI/preload (was Re: [RFC] improving memory mappingthru code exercising)

[Ed Phillips]

Yes, very cool Stas!

Perrin Harkins wrote:

> On Sat, 3 Jun 2000, Stas Bekman wrote:
>
> > correction for the 3rd version (had the wrong startup), but it's almost
> > the same.
> >
> >   Version Size   SharedDiff Test type
> >   
> > 1  3469312  2609152   860160  install_driver
> > 2  3481600  2605056   876544  install_driver & connect_on_init
> > 3  3469312  2588672   880640  preload driver
> > 4  3477504  2482176   995328  nothing added
> > 5  3481600  2469888  1011712  connect_on_init
>
> Cool, thanks for running the test!  I will put this information to good
> use...

Re: was apache.org down

[Ed Phillips]

Replying to myself.  It is back up, obviously. sorry for the noise



Ed Phillips wrote:

> Level 3 is broken.
>
> They know and are working on it. hmmm
>
> Ed

was apache.org down

[Ed Phillips]

Level 3 is broken.

They know and are working on it. hmmm

Ed

apache.org down

[Ed Phillips]

"Hughes, Ralph" wrote:

> COOL!
> I couldn't wait...
> I built and installed mod_perl 1.24 and it fixed the problem!   Now if I can
> just get the CGI module
> to recognize my domainname .. :-)
>
> -Original Message-
> From: Hughes, Ralph
> Sent: Friday, June 02, 2000 2:02 PM
> To: Geoffrey Young; 'Michael Todd Glazier'; ModPerl
> Subject: RE: Segmetation Fault problem
>
> I'm not too good on back traces myself.   `
> I'm using a dynamic build of mod_perl, so I may try building the 1.24
> version next week sometime.
> I hadn't thought of changing the PerFreshStart parameter, it might make a
> difference...
>
> -Original Message-
> From: Geoffrey Young [mailto:[EMAIL PROTECTED]]
> Sent: Friday, June 02, 2000 1:11 PM
> To: Hughes, Ralph; 'Michael Todd Glazier'; ModPerl
> Subject: RE: Segmetation Fault problem
>
> hmmm, did you try upgrading your installation then?
> you are using a static mod_perl?
> PerlFreshRestart Off?
>
> I'm no good at reading backtraces, but posting that is probably the next
> step (see SUPPORT doc section on core dumps in the distribution)
>
> sorry I can't be of more help...
>
> --Geoff

Re: oracle : The lowdown

[Ed Phillips]

For those of you tired of this thread please excuse me, but
here is MySQL's current position statement on and discussion
about transactions:

Disclaimer: I just helped Monty write this partly in response to
some of the fruitful, to me, discussion on this list. I know
this is not crucial to mod_perl but I find the "wise men who 
are enquirers into many things" to be one of the great things
about this list, to paraphrase old Heraclitus. I learn quite
a bit about quite many things by following leads and hints here
as well as by seeing others problems.

I'd love to see your criticism of the below either here or
off the list.


Ed
-


The question is often asked, by the curious and the critical, "Why is
MySQl not a transactional database?" or "Why does MySQl not support 
transactions."

MySQL has made a conscious decision to support another paradigm for 
data integrity, "atomic operations." It is our thinking and experience 
that atomic operations offer equal or even better integrity with much 
better performance. We, nonetheless, appreciate and understand the 
transactional database paradigm and plan, in the next few releases, 
on introducing transaction safe tables on a per table basis. We will 
be giving our users the possibility to decide if they need
the speed of atomic operations or if they need to use transactional 
features in their applications. 

How does one use the features of MySQl to maintain rigorous integrity 
and how do these features compare with the transactional paradigm?

First, in the transactional paradigm, if your applications are written 
in a way that is dependent on the calling of "rollback" instead of "commit" 
in critical situations, then transactions are more convenient. Moreover, 
transactions ensure that unfinished updates or corrupting activities 
are not commited to the database; the server is given the opportunity 
to do an automatic rollback and your database is saved. 

MySQL, in almost all cases, allows you to solve for potential 
problems by including simple checks before updates and by running 
simple scripts that check the databases for inconsistencies and 
automatically repair or warn if such occurs. Note that just by 
using the MySQL log or even adding one extra log, one can normally 
fix tables perfectly with no data integrity loss. 

Moreover, "fatal" transactional updates can be rewritten to
 be atomic. In fact,we will go so far as to say that all
 integrity problems that transactions solve can be done with 
LOCK TABLES or atomic updates, ensuring that 
you never will get an automatic abort from the database, which is a
common problem with transactional databases.
 
Not even transactions can prevent all loss if the server goes down.  
In such cases even a transactional system can lose data.  
The difference between different systems lies in just how small 
the time-lap is where they could lose data. No system is 100 % secure, 
only "secure enough". Even Oracle, reputed to be the safest 
of transactional databases, is reported to sometimes lose data
 in such situations.

To be safe with MySQL you only need to have backups and have the update
logging turned on.  With this you can recover from any situation that you could
with any transactional database.  It is, of course, always good to have
backups, independent of which database you use.

The transactional paradigm has its benefits and its drawbacks. Many users
and application developers depend on the ease with which they can code around
problems where an "abort" appears or is necessary, and they may have to do
 a little more work with MySQL to either think differently or write more.
 If you are new to the atomic operations paradigm, or more familiar or more
comfortable with transactions, do not jump to the conclusion that MySQL 
has not addressed these issues. Reliability and integrity are foremost 
in our minds.

Recent estimates are that there are more than 1,000,000 mysqld servers 
currently running, many of which are in production environments.  We hear
 very, very seldom from our users that they have lost any data, and in
 almost all of those cases user error is involved. This is in our 
opinion the best proof of MySQL's stability and reliability.

Lastly, in situations where integrity is of highest importance, MySQL's
 current features allow for transaction-level or 
better  reliability and integrity. 

If you lock tables with LOCK TABLES, all updates will stall until any
integrity checks are made.  If you only do a read lock (as opposed to
a write lock), then reads and inserts are still allowed to happen.
The new inserted records will not be seen by any of the clients
that have a READ lock until they relaease their read locks.
With INSERT DELAYED you can queue insert into a local queue, until
the locks are released, without having to have the client to wait for
the insert to complete.


Atomic in the sense that we mean it is nothing magical, it only means 
that you can be sure that while each specific up

Re: splitting mod_perl and sql over machines

[Ed Phillips]

>   > Most of my requests are served within 0.05-0.2 secs, but I afraid that
>   > adding a network (even a very fast one) to deliver mysql results, will
>   > make the response answer go much higher, so I'll need more httpd processes
>   > and I'll get back to the original situation where I don't have enough
>   > resources. Hints?

>   The network just has to match the load.  If you go to a switched 100M
>   net you won't add much delay.  You'll want to run persistent DBI
>   connections, of course, and do all you can with front-end proxies
>   to keep the number of working mod_perl's as low as possible.


Yes. Even under low load a 100M connected separate db server will bench
about the same performance as the intuitively "faster" unix socket connection
(for MySQL) on same box for reasons mentioned above. Special care can be taken to tune 
MySQL for the size of and type of requests as well, increasing
performance. See the MySQL docs.

I asked a question earlier about a client only(sep db) set up 
of DBD::Oracle and just which of the Oracle libraries are critical 
and which can be skipped. 

If you check the CLIENTS read me you'll see some contradictory advice. 
I'm curious about that because I'd love to minimize use of libraries
on Oracle set ups

not a mod_perl specific question so offlist would be great if any would 
be so kind.


ed

Re: APACHE_ROOT

[Ed Phillips]

Ged,

You are very entertaining. The code in question is also known as a combined
copy and substitution.

>Beware if you haven't got /src on the end of your source directory!

If you don't have a match with the string or regexp , you'll just get a straight copy.


Ed

   X-Authentication-Warning: C2H5OH.jubileegroup.co.uk: ged owned process doing -bs
   Date: Sat, 15 Jan 2000 00:00:37 + (GMT)
   From: "G.W. Haywood" <[EMAIL PROTECTED]>
   Content-Type: TEXT/PLAIN; charset=US-ASCII
   Sender: [EMAIL PROTECTED]
   Precedence: bulk

   Hi there,

   On 14 Jan 2000, William P. McGonigle wrote:

   > Can someone explain what APACHE_ROOT is meant to be?  I'm assuming
   > it's somehow different thatn APACHE_SRC (which I'm defining).

   The expression

   ($APACHE_ROOT = $APACHE_SRC) =~ s,/src/?$,,;

   sets the scalar $APACHE_ROOT to be equal to the scalar $APACHE_SRC and
   then chops off any "/src" or "/src/" from the end of it.  
   

   The =~ binding operator (p27) tells perl to do the substitution
   s,/src/?$,, to the thing on left hand side of its expression.

   The parentheses (p77) mean the thing in them is a term, which has the
   highest precedence in perl so the assignment has to be done first.

   The substitution then has to be done on the result, $APACHE_ROOT and
   not $APACHE_SRC, er, obviously.

   The three commas are quotes (p41) for a substitution, presumably
   chosen because they can't easily appear in a filename.

   The pattern to match is

   /src/?$

   The question mark is a quantifier (p63), it says we can have 0 or 1
   trailing slash in the pattern we match - it's trailing at the end of
   a string because of the $ (p62).

   If our string matches, the matching bit is replaced with the bit
   between the second and third commas.  There's nothing between the
   second and third commas, so it's replaced with nothing.  Have a look
   at pages 72 to 74 especially for more about the s/// construct.

   The page numbers are from the Camel Book, second edition.  I keep it
   on my desk at all times, it stops my papers blowing around.  You will
   help yourself a lot with these things if you read chapters one and two
   five or six times this year as a kind of a penance.

   So if

   $APACHE_SRC eq  "/usr/local/apache/src/"

   or

   $APACHE_SRC eq  "/usr/local/apache/src"

   then

   $APACHE_ROOT eq "/usr/local/apache"

   after the substitution.

   I just *love* Perl's pattern matching!

   73,
   Ged.

Re: modperl success story

[Ed Phillips]

>The troll vanisheth!

ha!

Reminds me of the Zen story of an old fisherman in a boat on a lake in a heavy can't 
see your hands fog. He bumps into another boat, and shouts at the other guy, "Look 
where you're going would you! You almost knocked me over."  He pulls up beside the 
boat and is about to give the other guy a piece of his mind, but when he looks in the 
other boat, he discovers that no one else is there.

Flame trolls on mailing lists are virtual empty boats, whose only value is the 
sometimes humorous apoplexy elicited in the old sea salts on the list.


Ed

Re: mysql.pm on Apache/mod_perl/perl win98

[Ed Phillips]

Hi Dave,

I only do *nix, but I think that you should not need mysql.pm if you are using
DBI/DBD. Jochen is quite helpful on the MySQL modules list. subscription info availble 
at www.mysql.com.

Good Luck,

Ed

Re: Comparing arrays

[Ed Phillips]

Cliff,

I wanted him to work for the rest of it, or at least go to another list.

It looks like he wanted two arrays, @in_hash_one_alone and @in_hash_two_alone,
so having him push to one array may confuse him. he's better off doing a little
studying, methinks.

ed

Re: Comparing arrays

[Ed Phillips]

Really Dheeraj,

This is not a mod_perl specific question, and I don't know the all important context 
into which this boilerplate code you are seeking to elicit from the list is to be 
dropped.

here is a boilerplate "find me keys that are not in both hashes":

foreach (keys %hash_one) {
  push(@here_not_there, $_) unless exists $hash_two{$_};  
}

shame on you. To expiate your sins, read perldoc pages for two hours
everyday for two weeks.

ed

Re: DBI

[Ed Phillips]

This is also not a mod_perl question.

depending on where your DBD::Oracle is installed you can get away with certain 
liberties in the Oracle library department. 

Nonetheless, you should continue your inquiry on a DBI related list.

Thank you,

Ed

Re: Server Stats

[Ed Phillips]

>this is like closing the gate after the horse has bolted without things
>like decent locking and transactions. Although perhaps I'm mistaken and

You can rest assured that they know what they are doing. :-)

It is also worth upgrading to newer versions. The newest versions not deemed stable 
just yet no longer use ISAM, are much faster, and will allow for a host of new 
features. stay tuned.

ed

Re: Spreading the load across multiple servers (was: Server Stats)

[Ed Phillips]

>I don't have any real answers - just a suggestion. What is wrong with the
>classic RDBMS architecture of RAID 1 on multiple drives with MySQL - surely
>it will be able to do that transparently?


Yes, RAID is very helpful with MySQL.  I spoke with Monty, the developer of MySQL at 
the open source conference in Monterey and he said that they are currently working on 
replication and mirroring features. It might be worth inquiring directly with them. 


Ed

Re: Apache::DBI & MySQL

[Ed Phillips]

Even with MySQL, persistent connections are a performance boost, although you need to 
have a reason to use them. That is, only open a persistent connection when you have 
determined that you will use that connection enough to make a difference. Other 
connections may be better, as Ken said, one-time.

Ed