ANNOUNCE: Apache::SMTP

[mock]

A while ago I attended an interesting talk by Stas at the Vancouver pm.  The
idea of building an SMTP server using mod_perl was mentioned and the task kind
of appealed to me, so here's my first attempt at it.  Note that it uses
Apache::TieBucketBrigade, which is my attempt at making bucket brigades look
like IO::Handles.  This may or may not be a good idea; I'd appreciate some
feedback.

BTW, I'm trying to figure out how to test both of these modules using 
Apache::Test, but I'm not having a lot of luck.  Any pointers would be 
helpfull.

thanks, and enjoy my sketchy code ;)

code can be found here: http://search.cpan.org/~mock/

mock

-- 
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html

Re: ANNOUNCE: Apache::SMTP

[mock]

On Mon, Feb 23, 2004 at 11:27:33AM -0800, Stas Bekman wrote:
> Cool. It looks nice, but you shouldn't have rushed to release things on 
> CPAN, IMHO.
> 
> Why taking over the Apache::SMTP namespace if you don't implement a full 
> SMTP protocol? It's usually a good idea to post a proposal here so we may 
> help to choose a suitable name. If you badly want to put things on CPAN, 
> include _ in the version number so that the PAUSE indexer won't index it.
> 
> Besides taking over Apache::SMTP, it should probably live somewhere under 
> Apache::Protocol::. For some ideas see:
> http://perl.apache.org/products/apache-modules.html#Module_Naming_Conventions

It should be a full implementation of SMTP, if it isn't then there's a bug.
What it doesn't do, is proper queuing, any sort of real validation of the
sender or recipient, domain specific configuration, or local delivery.  My 
thought for the namespace was that those sorts of things would be implemented
in subclasses - Apache::SMTP::LocalDelivery - and the like.  I'm not sure I
like the Apache::Protocol:: namespace as it, to my mind at least, doesn't 
capture the idea that it implements a daemon.  I toyed with 
Apache::Net::Server::Mail::SMTP since it really is just Net::Server::Mail::SMTP
bolted on to a handler, but that name space is just fugly ;).  I suppose it
could be Apache::Server::SMTP or perhaps Apache::Daemon::SMTP but both of those
seem somehow redundant; I mean, "Apache::Server?" Apache is a Server.  Anyway
after agonizing over the namespace for quite some time I decided to just go
with simple and to the point Apache::SMTP (although I thought about putting
a D on the end for Apache::SMTPD).  Anyway, obviously I ate too much lead paint
as a child, however I'm certainly willing to move to a different name space
if someone thinks there'll be others wanting to camp out in Apache::SMTP.

> 
> As for Apache::TieBucketBrigade, it's probably a good idea to make it a 
> part of the mp2 core and implement it in C. There is a plan to provide TIE 
> API for filters as written down at modperl-2.0/todo/features_new

Well, I think I've kinda proven it would be usefull.  Many of the things I
did are kinda hackish in perl.

> 
> >BTW, I'm trying to figure out how to test both of these modules using 
> >Apache::Test, but I'm not having a lot of luck.  Any pointers would be 
> >helpfull.
> 
> http://perl.apache.org/docs/general/testing/testing.html
> http://www.perl.com/pub/a/2003/05/22/testing.html
> modperl-2.0 test suite and several other Apache:: modules on CPAN which now 
> include a test suite.
> 
> You want to keep TestTieBucketBrigade.pm out of project/lib and put it 
> under t instead. See the modperl-2.0 test suite.
>

I'm having a little difficulty figuring out how to create request which isn't
HTTP.  I'll move TestTieBucketBrigade and rerelease just as soon as I figure
out how to make Apache::TestRequest do what I want.

thanks

mock

-- 
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html

Re: ANNOUNCE: Apache::SMTP

[mock]

On Mon, Feb 23, 2004 at 03:38:17PM -0800, Stas Bekman wrote:
> Geoffrey Young wrote:
> 
> >>By commenting on Apache::TieBucketBrigade I was trying to
> >>reserve that or a similar namespace for the modperl core. So later there
> >>would be no collision/confusion.
> >
> >oh, I didn't get that from your comment.  anyway, if something like that
> >ever makes it into core it probably ought to be in the APR namespace, so I
> >guess we're safe :)
> 
> Yup, which also shows that the Apache::TieBucketBrigade name is a wrong 
> name, since it's an APR feature and not Apache's one.
> 
> But if it's already implemented, nothing stops us from putting it into the 
> core as it is (in perl) and then port it to C when we get time. Or may be 
> 'mock' will want to do that ;)
> 

I was about to release a new version of TieBucketBrigade with a proper test
suite, now that I've kinda figured out Apache::Test (thanks for the pointers
Stas, I knew I was missing something).  Is there a different namespace I 
should use? or shall I just go ahead and drop the code and hope it all sorts
itself out later?  As for a C port, I may get to that if I have enough 
time + motivation, but it's a little outside the scope of the project
I'm working on, so I don't know if I'll get to it.  I'll race you ;).

Speaking of namespace, unless someone comes up with something better, I'm going
to continue using Apache::SMTP.  I'm reasonably ok with Apache::Protocol::SMTP,
or something of similar size, but any deeper than that and it's going to get
stupid when I try to attach something like Mail::Audit to it.
Apache::Net::Server::Mail::SMTP::Mail::Audit::PGP or something of the like is
just aesthetically sucky.  So, suggestions please?

mock

-- 
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html

Re: ANNOUNCE: Apache::SMTP

[mock]

On Tue, Feb 24, 2004 at 09:35:57AM -0800, Stas Bekman wrote:

> You probably want to talk to Geoff, since he said he has already started 
> working on it. May be he has enough dough, so that you can finish it off. 
> And as he said it'll will in the APR:: namespace, so there is no collision. 
> I suppose once the core supports it, you can just nuke it from CPAN if you 
> are happy with it.

This works for me.

> >stupid when I try to attach something like Mail::Audit to it.
> >Apache::Net::Server::Mail::SMTP::Mail::Audit::PGP or something of the like 
> >is
> >just aesthetically sucky.  So, suggestions please?
> 
> That's exactly where the problem is with your name choice. If you plan to 
> add things which are aren't Net::SMTP, then the name is wrong. As I 
> suggested it looks more like an application than a framework or protocol 
> API. It just happens to work only Apache. Therefore Apache::SMTP is 
> inappropriate. Usually when you write an application you give it some name, 
> not after the protocol but something else, usually unique and cool of 
> possible. e.g. Apache::MockMail. And several people have suggested to put 
> new Apache App modules into the Apache::App:: namespace, but it's up to you 
> to decide.

You raise a good point.  The App space seems like it would be more 
appropriate.  MockMail, seems a bit pretensious ;), however I think your
correct, SMTP is perhaps not right.  How about Apache::App::SMTP?  Can
everyone else live with that?  Alternatively, Apache::App::SMTPd.

> 
> Meanwhile, I've linked to your module from:
> http://perl.apache.org/docs/2.0/user/handlers/protocols.html#CPAN_Modules
> as the first mp2 protocol implementation on CPAN, that I know of.

cool

mock

-- 
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html

Re: ANNOUNCE: Apache::SMTP

[mock]

On Tue, Feb 24, 2004 at 02:29:16PM -0500, Geoffrey Young wrote:
> 
> > You probably want to talk to Geoff, since he said he has already started
> > working on it. May be he has enough dough, so that you can finish it
> > off. And as he said it'll will in the APR:: namespace, so there is no
> > collision. I suppose once the core supports it, you can just nuke it
> > from CPAN if you are happy with it.
> 
> mock is much farther along with it than I am - I was creating it as a
> tutorial for people who get a $bb and don't know what to do with it (as with
> Apache::Upload).  but they look surprisingly similar :)
> 
> the one thing I was focusing on before I got sidetracked was support for $/
> - specifically if $/ is undef letting READLINE use $bb->flatten to return
> the entire brigade, but also splitting the

This is a good idea... I'll see what I can do with my bits.

> 
> another thing I was debating was how to treat the brigade when reading - you
> seem to be treating the brigade like you would for Apache filters,
> destroying the brigade while reading.  I wanted more of a filehandle
> interface where the brigade could be manipulated over and over again
> (reading a file is non-destructive, as is APR::Brigade::flatten.
>

I think I need to read the Brigade docs again :).  But, yeah, when I was 
building the thing, I was kinda thinking about it like it was a filter.  
Your approach sounds like it might be better though.
 
> anyway, the (very, very rough) work in progress is here
> 
> http://www.modperlcookbook.org/~geoff/modules/experimental/APR-TieBrigade-0.01.tar.gz
> 
> debug statements and other ugliness included.  but it might trigger some
> ideas you had, or give you an idea as to how better use Apache-Test.  for
> what it's worth :)
>

Cool, I'll take a look and see what can be done.
 
> anyway, I asked [EMAIL PROTECTED] to add an APR:: namespace to CPAN a while
> ago, but they might not have gotten around it it yet.  however, before
> releasing to that namespace I'd consider a name that more specifically suits
> the intended purpose - if indeed the module is only relevant to brigades use
> in input/output filters, where you need to destroy the incoming and write to
> a new outgoing, then I'd probably either keep it in the Apache:: namespace
> to name it something like APR::TieFilterBrigade.  also note that there
> should be _zero_ mod_perl dependencies if placed in the APR:: namespace (I
> wasn't quite there either :)

I'll try and fix it up so it isn't destructive (if I can, and still make it
work the way I need it to).  I'm not sure if I can pull out all the mod_perl
dependencies though.  If that is the case it should stay in the Apache 
namespace, right?

thanks

mock

-- 
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html

Re: APR::Base64 uses

[mock]

On Tue, Aug 10, 2004 at 08:57:14PM -0400, Geoffrey Young wrote:
> 
> >>but hiding the decryption key from technical people is generally
> >>impossible
> > 
> > 
> > Only if they crack your application server.  Cracking the database or
> > sniffing packets would not be good enough, assuming traffic to your
> > credit card company is over SSL.
> 
> oh, sure.
> 
> I guess I had a different mindset with all of that - internal employees.
> for most big companies I would assume that accessing the underlying Oracle
> financials database (or some other "enterprise" solution) with _all_ the
> credit card numbers would be sufficiently difficult for outside hackers.
> I'd be much more worried about the disgruntled employee causing trouble.
> 
> but you're right - crackers are a legitimate concern for this kind of thing,
> and I wasn't aware of the role that verisign is now playing (which I guess
> is the cost of having worked someplace where we built everything from
> scratch).  so, thanks for the knowledge :)
> 
> --Geoff
> 
> -- 
> Report problems: http://perl.apache.org/bugs/
> Mail list info: http://perl.apache.org/maillist/modperl.html
> List etiquette: http://perl.apache.org/maillist/email-etiquette.html
> 

This is about to go out to CPAN, but since it seems there is some question
as to how to do this, I'll send it out a little early.  Attached is a module
for safely encrypting and storing credit cards using the Business::OnlinePayment
interface.  It uses a public RSA key and Crypt::OpenSSL::RSA to encrypt a 
randomly generated Blowfish symetric key, which is used with Crypt::CBC to
encrypt the creditcard details.  Also included is a module for reversing the
transaction given the RSA private key.  This allows one to store creditcards
safely on the server, by not putting the private RSA key on the same machine.
To retrieve the cards, simply pull the data to an offline machine, and 
decrypt using the RSA private key.  Assuming you aren't an idiot and leave
the private key somewhere internet accessible, this should be as proof against
hackers as any other credit card processing.

enjoy

mock


Business-OnlinePayment-StoredTransaction-0.01.tar.gz
Description: application/tar-gz
-- 
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html

Re: APR::Base64 uses

[mock]

On Wed, Aug 11, 2004 at 01:01:23AM -0700, Bart Simpson wrote:
> 
> --- mock <[EMAIL PROTECTED]> wrote:
> 
> > This is about to go out to CPAN, but since it seems
> > there is some question
> > as to how to do this, I'll send it out a little
> > early.  Attached is a module
> > for safely encrypting and storing credit cards using
> > the Business::OnlinePayment
> > interface.  It uses a public RSA key and
> > Crypt::OpenSSL::RSA to encrypt a 
> > randomly generated Blowfish symetric key, which is
> > used with Crypt::CBC to
> > encrypt the creditcard details.  Also included is a
> > module for reversing the
> > transaction given the RSA private key.  This allows
> > one to store creditcards
> > safely on the server, by not putting the private RSA
> > key on the same machine.
> > To retrieve the cards, simply pull the data to an
> > offline machine, and 
> > decrypt using the RSA private key.  Assuming you
> > aren't an idiot and leave
> > the private key somewhere internet accessible, this
> > should be as proof against
> > hackers as any other credit card processing.
> > 
> > enjoy
> > 
> > mock
> > 
> 
> This sounds really awesome. Thanks. just what we need.
> 
> Can one key decrypt the whole batch of cards?
> 

Yes, you only need one RSA public/private key pair to decrypt everything.
There's a script included in the tarball to make the key pair.

mock


-- 
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html

Re: APR::Base64 uses

[mock]

On Wed, Aug 11, 2004 at 08:23:42AM -0500, JupiterHost.Net wrote:
> 
> >
> >This sounds really awesome. Thanks. just what we need.
> >
> >Can one key decrypt the whole batch of cards?
> 
> I didn't look to close but I'm thinking "no" since it was randomly 
> generated, and a single key for them all would again make it pointless 
> to do anything with it because all they need now is one piece of data to 
> see all the CC info. Instead of one per record.
> 
> Just my .02 ;p
> 
> Lee.M - JupiterHost.Net
> 

The blowfish keys are randomly generated for each transaction, but the RSA keys
remain the same.  Business::OnlinePayment::StoredTransaction::Unstore
uses the RSA private key to decrypt each blowfish key, then uses that blowfish
key to decrypt the transaction.  Thus, as far as the module user is concerned
the keys remain the same.  

If you want something that uses a separate key for each transaction, I have
another module which effectively accomplishes this, but it's not ready for
production yet.  I'm not sure why you'd want this though, as it becomes a
key management nightmare.

BTW, I will be talking about a bunch of new tricks for credit card processing
at YAPC::Europe this year, as well as, (just to bring the topic back to
mod_perl) the experience I had building an MTA using Apache and mod_perl.

mock

-- 
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html

Re: APR::Base64 uses

[mock]

On Wed, Aug 11, 2004 at 09:36:42AM -0400, Perrin Harkins wrote:
> mock wrote:
> >This is about to go out to CPAN, but since it seems there is some question
> >as to how to do this, I'll send it out a little early.  Attached is a 
> >module
> >for safely encrypting and storing credit cards using the 
> >Business::OnlinePayment
> >interface.
> 
> Public key encryption is the best that can be done here, but even so, if 
> a cracker compromises your machine, he can just add a "warn $cc_number" 
> in your code before you encrypt it.  Your old cards will be safe though.
> 
> - Perrin

It's (almost) exactly equivalent to any other credit card processing.  The
blackhat in question could add a warn statement to any credit card processing
(unless it's a third party payment system like PayPal).  The only other
risk with this system is that if you lose your private key then the security
is compromised.  This really isn't an additional risk, as the credit card
processors you are relying on have the exact same problem.  There is the
additional problem with external processors, in that they are often 
vulnerable to man in the middle attacks, which if I was a h/cracker (which I
suppose I am at times -- come to our security conference in Tokyo in November
http://www.pacsec.jp -- shameless plug) would be the first place I'd attack.

As you said though, your old cards will be safe.

mock

-- 
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html

Re: How to tell if there's data waiting on a NON-blocking Apache connection?

[mock]

On Wed, Aug 18, 2004 at 02:09:53PM -0700, Stas Bekman wrote:
> Glenn Strauss wrote:
> >On Wed, Aug 18, 2004 at 12:59:51PM -0700, Ken Simpson wrote:
> >
> >>>the APR::Socket object is an opaque one, so it can't interoperate with 
> >>>any other perl modules. Have you looked if there is some C api to get 
> >>>the native socket object? There could be one (as they have for file 
> >>>objects), I didn't check.
> >>
> >>I looked through apr_network_io.h, which seemed like the logical
> >>place, and couldn't find an API that returns the native socket
> >>object. But I'm pretty unfamiliar with the Apache code base, so
> >>someone else would probably have better luck.
> >
> >
> >This is what we'd need bound to the Perl API:
> >
> >apr_os_sock_t fd;
> >apr_os_sock_get((apr_os_sock_t *) &fd, (apr_socket_t *) client_socket);
> >
> >On Unix-type platforms, apr_os_sock_t is an int -- the file descriptor,
> >which you can use with select() or IO::Select() or anything you
> >like in Perl to poll the descriptor:
> 
> and what do you do with socket fd to get it to work with IO::Select?
> 
> >$rin = '';
> >vec($rin, $fd) = 1;  ## $fd directly instead of fileno(STDIN) or 
> >fileno($FH)
> >...
> >$nfound = select($rout=$rin, undef, undef, $timeout);
> >
> >
> >(On other platforms, like Windows, I don't know what apr_os_sock_t is;
> > check the headers files. :)
> 
> I'd rather not expose OS specific bits if they won't work with all perl 
> modules. APR provides the API that should (hopefully) work on all 
> platforms, so why not use that?
> 
> >To get the client socket for a connection, you can obtain the
> >(apr_socket_t *) with the incantation:
> >
> >apr_socket_t *client_socket =
> >  ap_get_module_config(r->connection->conn_config, &core_module);
> >
> >and then use apr_os_sock_get() to get the fd.
> 
> Yup, we already have that one.
> 
> 

Currently we use the following hacky solution to make the fd work with 
IO::Select and Core::select

use constant MAGIC_FILENO => 7;

use ex::override
GLOBAL_select => sub {
 if (@_ == 1) {
 my $sh = shift;
 unless (ref($sh)) {
 my $caller = caller();
 $sh = \*{$caller .'::'. $sh};
 }
 return CORE::select();
 }
 elsif (@_ == 4) {
 my $rin = vec($_[0],MAGIC_FILENO,1) if $_[0];
 my $win = vec($_[1],MAGIC_FILENO,1) if $_[1];
 my $ein = vec($_[2],MAGIC_FILENO,1) if $_[2];
 if ($rin or $win or $ein) {
 #magic fileno do something special
 #should poll the socket for data here,
 #but that doesn't work yet, so fake it
 return wantarray ? (1,$_[3]) : 1;
 }
 else {
 return
 CORE::select($_[0],$_[1],$_[2],$_[3]);
 }
 }
 else {
die "WTF ?";
 }
 };


In the latest incarnation of TieBucketBrigade we do the following

sub FILENO {
#returns magic fileno which select and fdopen will do something special with
shift;
return MAGIC_FILENO;
}

Provided you don't need fd 7 anywhere else, this should work.  Ideally we 
should get the real fd and use it instead.

So all we need to make IO::Select, select and everything else work, is the
ability to poll for data on the socket, which we'll just stick in the 
overridden select at the appropriate point above.

mock

-- 
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html

Re: ANNOUNCE: Apache::Qpsmtpd

[mock]

On Tue, Sep 21, 2004 at 07:19:48PM +0100, Matt Sergeant wrote:
> On 19 Sep 2004, at 10:34, Matt Sergeant wrote:
> 
> >Last week I was at YAPC and attended mock's talk about building an 
> >SMTPd in mod_perl2/apache2. But before going I wanted to be able to 
> >build something like that for myself. So I wrote Apache:Qpsmtpd - a 
> >mod_perl2 handler that embeds Qpsmtpd (http://smtpd.develooper.com/)
> >
> >  http://www.sergeant.org/Apache-Qpsmtpd/
> >
> >It seems to mostly work, though if you want to play with it you'll 
> >need the current cvs of qpsmtpd. It's about 3 times faster than the 
> >qpsmtpd-forkserver that ships with qpsmtpd.
> 
> Someone posted privately asking why I thought it was faster, but my 
> reply to them bounced (broken anti-spam system I think) so I'll reply 
> here:
> 
> I think it's faster because it pre-forks. qpsmtpd-forkserver is just 
> naive - it forks once per connection. Plus I guess Apache does the I/O 
> for you, though I don't think that's really gaining me much - consider 
> the overheads of the bucket brigade vs Perl's readline.
> 
> Matt.
> 

I think there is the potential for an even better speed increase with a little
reworking of how QPSMTPD works.  Some of the stuff you're doing (the timeout
bits you showed me) might be better left to Apache to handle.  Also, you might
want to try it with the threading MPM vs the preforking.  Some OS's see a bit
of improvement.  Any idea where your bottleneck is?

mock

-- 
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html

Re: Fwd: XSS evasion

[mock]

On Fri, Oct 06, 2006 at 07:25:06PM +0200, Clinton Gormley wrote:
> On Fri, 2006-10-06 at 18:48 +0200, Hendrik Van Belleghem wrote:
> > "mock" talked about XSS at this years YAPC::Europe in Birmingham a few
> > weeks ago. He had quite a few examples. His slides are at
> > http://sketchfactory.com/static/mvc.pdf (More Vulnerable Code).
> > It goes without saying that it would be a bit unwise to test the URLs
> > mentioned in the talk.
> 
> He briefly mentions HTML::Scrubber in there. I am using
> HTML::Stripscripts::Parser, which also makes sure that tags are nested
> properly.
> 
> Anybody have any view on these (or other) modules?
> 
> Clint
> 

HTML::Scrubber is not really broken.  The problem is that the
documentation leads the user to do broken things, as was shown with
Planet Plagger.  It is possible to make a secure HTML::Scrubber config,
but you need to default deny everything and then only allow a select
list of tags and attributes - and you need to really think about that
list.  The underlying problem, which I suspect HTML::Stripscripts shares
is that HTML::Parser thinks that the attribute "foo=bar" is different
than the attribute "foo.=bar" (RSnake covers this kind of evasion in his
document fairly well) and your browser thinks that everthing non
alphanumeric before the equals sign is junk.  So without actually
sitting down and auditing HTML::Stripscripts I'd say it probably _can_
be used safely, but odds are most people won't.

As an aside, I'm not sure that it's completely public knowledge outside
people who read my site and people who saw my talk, but CSRF can be
performed from feeds (RSS, Atom).  So you need to defang any feeds that
will be shown to a logged in user otherwise there's the possibility of
doing various bad things(tm).  There's also the obvious problem with
javascript in feeds as well...

Finally, if you're near Tokyo this November, Martin Johns is going to be
presenting what looks like a really good talk on CSRF, with some new
mitigation techniques and a fair bit of new work on the problem at
PacSec (http://pacsec.jp/speakers.html).

mock

Catchall ScriptAlias and error handling

[Andreas Mock]

Hi all,

I searched the internet for a wile now and couldn't find an answer to my 
question.
We have the following situation:

a) In a virtual host configuration we have several Alias entries pointing to 
several 
directories and files
b) At the end of the config we have a ScriptAliasMatch entry of the form
ScriptAliasMatch ^(.*)$ /some/directory/doit.pl$1
and  

SetHandler perl-script
PerlResponseHandler ModPerl::Registry
PerlOptions +ParseHeaders
Options +ExecCGI


It works as expected in regards to the "catchall" behaviour of the script.
BUT: We would like the following. As soon as a file in one of the aliased 
directories is not found
the server should return the standard error message. At the moment a subrequest 
hits the perl
handler doit.pl.

Now my question: How can I achieve the desired behaviour?
Is there a way to internally redirect to the apache standard error handler?
How can I identify these kinds of subrequests?

Best regards
Andreas 

ModPerl::Registry and custom error documents

[Andreas Mock]

Hi all,

I searched and googled around for a while but couldn't find a solution for
the following question.

We're using ModPerl::Registry (PerlOptions +ParseHeaders)
in combination with mod_perl to serve our pages in a CGI like
environment. This has the advantage for us to have only a very very
small layer of integration between the scripts an the special runtime
environment.

Now we want to serve custom made error documents and thought
naively to get it work by just setting the status to the error code
and serving a html-page as it works with a normal status 200 page.

But this doesn't work. It seams that apache jumps right into the
error handling stuff as soon as a status code >= 400 is seen.

How can we dynamically create own error documents without using
the lower level mod_perl/apache api? Is there a simple way?
How can we achieve that?

Best regards
Andreas Mock

ModPerl::Registry and custom error documents

[Andreas Mock]

Hi all,

I searched and googled around for a while but couldn't find a solution for
the following question.

We're using ModPerl::Registry (PerlOptions +ParseHeaders)
in combination with mod_perl to serve our pages in a CGI like
environment. This has the advantage for us to have only a very very
small layer of integration between the scripts an the special runtime
environment.

Now we want to serve custom made error documents and thought
naively to get it work by just setting the status to the error code
and serving a html-page as it works with a normal status 200 page.

But this doesn't work. It seams that apache jumps right into the
error handling stuff as soon as a status code >= 400 is seen.

How can we dynamically create own error documents without using
the lower level mod_perl/apache api? Is there a simple way?

Best regards
Andreas Mock

ModPerl::Registry and Error Documents

[Andreas Mock]

Hi all,

after searching for a while without finding an answer to my question
I hope you can help me,

We're using mod_perl with ModPerl::Registry to have only a thin
layer between apache and our perl scripts. So far so good.
Now we want to produce error documents dynamically and thought
this could be achieved by simply producing a normal html document
and setting a status code > 400.

But it seems that the apache error handling jumps in as soon as
the code is seen and the html document is not delivered to 
the client.

My question: Which is an easy and correct way to produce ErrorDocuments while
using ModPerl::Registry? 

Best regards
Andreas

Re: ModPerl::Registry and Error Documents

[Andreas Mock]

> -Ursprüngliche Nachricht-
> Von: "Torsten Foertsch" 
> Gesendet: 29.09.09 10:01:26
> An: modperl@perl.apache.org
> Betreff: Re: ModPerl::Registry and Error Documents

> 
> You have asked this question already 2 times before, on 17.09.09 12:13 
> and 17.09.09 11:15. The first time I have answered it.
> 
> If my answer is not satisfying could you be so kind and explain why?

Hi Torsten, hi all others,

I have to apologize for sending the same question for many times.
I had problems with my mailer which let me think that my posts
never arrived at the mailing list. That was also the reason for
registring with another account, sending the question again.

Sorry for bothering.

Asking again was NOT a sign of being unsatisfied with an answer.
Sorry Torsten for giving you that feeling.

It was just me interpreting the outcome of an error the wrong way.
Shame on me.


> 
> Und übrigens, der Text in meiner Signatur ist ernst gemeint. Ich bin 
> käuflich.

Wer (im weitesten Sinne) nicht...  ;-)

Best regards
Andreas

Re: ModPerl::Registry and custom error documents

[Andreas Mock]

> -Ursprüngliche Nachricht-
> Von: "Torsten Foertsch" 
> Gesendet: 17.09.09 13:14:16
> An: modperl@perl.apache.org
> CC: Andreas Mock 
> Betreff: Re: ModPerl::Registry and custom error documents


> On Thu 17 Sep 2009, Andreas Mock wrote:
> > How can we dynamically create own error documents without using
> > the lower level mod_perl/apache api? Is there a simple way?
> > How can we achieve that?
> 
> 
> Then you can configure ErrorDocuments, e.g.
> 
> ErrorDocument 404 http://huhu.com
> ErrorDocument 500 "dumm gloffe"

Hi Torsten, hi all,

ok, this seems to be the simple case.
Thank you for this answer.

Is there another way to create custom ErrorDocuments based
on the return value of a ModPerl::Regisry script as long as the
script has not printed to stdout yet?

Best regards
Andreas

RE: Setting Content-Type in a custom response (mp2)

[Andreas Mock]

Hi all,

I searched for that solution for hours not finding any until
Torsten posted this one.

IMHO it's worth it to put Torsten's snippet to the documentation
pages of mod_perl.

Even the book 'mod_perl 2 user's guide' leaves a big hole
in that area.

Thank you Torsten!

Best regards
Andreas Mock



-Ursprüngliche Nachricht-
Von: Steve Hay 
Gesendet: 12.03.2010 10:58:15
An: "Torsten Förtsch" ,modperl@perl.apache.org
Betreff: RE: Setting Content-Type in a custom response (mp2)

>Torsten Förtsch wrote on 2010-03-11:
>> On Thursday 11 March 2010 15:18:08 Steve Hay wrote:
>>> I have a mod_perl-2 handler that uses custom_response() to display
>>> error messages if something goes wrong:
>>> 
>>> $r->custom_response(Apache2::Const::SERVER_ERROR, $error);
>>> return Apache2::Const::SERVER_ERROR;
>>> 
>>> That almost works fine, but the trouble is that the Content-Type header
>>> is always set to "text/html; charset=iso-8859-1", in which the charset,
>>> at least, is potentially wrong: all the pages in my software produce
>>> UTF-8 and are normally output with Content-Type "application/xhtml+xml;
>>> charset=utf-8".
>>> 
>>> How do I set the Content-Type when issuing a custom_reponse()? The
>>> usual thing of calling
>>> 
>>> $r->content_type("application/xhtml+xml; charset=utf-8");
>>> 
>>> doesn't seem to work in this case.
>>> 
>> I think that is impossible. ap_send_error_response() contains this code:
>> 
>> if (apr_table_get(r->subprocess_env,
>>   "suppress-error-charset") != NULL) {
>> core_request_config *request_conf =
>> ap_get_module_config(r->request_config,
>> &core_module);
>> request_conf->suppress_charset = 1; /* avoid adding default
>>  * charset later
>>  */
>> ap_set_content_type(r, "text/html");
>> }
>> else {
>> ap_set_content_type(r, "text/html; charset=iso-8859-1");
>> }
>>  The resulting content-type is either "text/html; charset=iso-8859-1" or
>> "text/html".
>> 
>> But instead of passing the response text directly to $r-
>>> custom_response you
>> can specify an URL. ap_die() will then create an internal redirect to
>> this URL. Via $r->prev->pnotes you can then access the pnotes of the
>> failed request. Putting all together a solution could look like:
>> 
>>   $r->pnotes->{estring}=$error;
>>   $r->pnotes->{ect}='my/special-content-type'; # use an otherwise not
>>   used URL here $r->custom_response(Apache2::Const::SERVER_ERROR,
>>   '/-/error');
>>   
>>   
>> SetHandler modperl
>> # please insert a \ at the end of each of the following lines
>> # up to }"
>> PerlResponseHandler "sub {
>>   my ($r)=...@_; return Apache2::Const::NOT_FOUND unless $r->prev;
>>   $r->content_type($r->prev->pnotes->{ect});
>>   $r->print($r->prev->pnotes->{estring}); return Apache2::Const::OK;
>> }"
>>   
>> 
>
>Thanks! That works a treat :-)

Re: Need help with abort connection

[Andreas Mock]

Hi Tobias,

your whole scenario depends on the possibility to reliably
cancel a running query on the database.

If your database and client (software) is able to do so,
which has to be tested, the cancel button could send
another ajax request with an identifier of the initial
query which then would be canceled on the server side.

That on the other handside would mean that your initial
query request would put the query in a queue with an id,
send the id back immediately, redirecting the client to
an response page wich would present the query result
when available. Beware of TCP-IP timeouts, so that the
resultpage should redirect to itself regularly.

Best regards
Andreas



-Ursprüngliche Nachricht-
Von: Tobias Wagener 
Gesendet: 22.07.2010 12:34:50
An: modperl@perl.apache.org
Betreff: Need help with abort connection

>Hello,
>
>I'm using 
>  > apache 2.2.3 
>  > mod_perl   2.0.2
>  > mod_apreq2 2.08
>on a linux box.
>
>In my enviroment a user can submit a query to a database. This request
>is handled as an Ajax-request. If he forget to add some search parameters
>this query can run a long time. So I add a "Cancel"-Button to the frontend
>that abort the request. That will work with no problems. But on the
>server side the request/query keeps on running.
>
>My handler first collect all data and then send the header and print
>the content. Because the handler works for Ajax and "normal" Requests
>and first after finishing the complete request I can decide if the
>"Content-Type" is html/pdf/json/xml/etc.
>
>Can someone give me a hint or better a solution how I can abort the
>request on the server side, too.
>
>Please excuse my english, but I hope you understand my question ;-)
>
>Best regards,
>
>Tobias
>

RE: Mod Perl - Custom Error Pages

[Andreas Mock]

Hi David,

there is
http://www.gossamer-threads.com/lists/modperl/modperl/100379?search_string=error%20page%20mock;#100379

thanks to Torsten.

And here is a snippet, I use

...
elsif($response->status == Apache2::Const::HTTP_NOT_FOUND) {
$r->status($response->status);
# Cookies trotzdem mitschicken
foreach my $cookie ($response->headers->header('Set-Cookie')) {
$r->err_headers_out->add('Set-Cookie' => $cookie);
}
if($response->body()) {
$r->subprocess_env('suppress-error-charset' => 1);
$r->content_type($response->headers->header('Content-Type'));
$r->err_headers_out->add('Content-Type' => 
$response->content_type());
$r->custom_response(Apache2::Const::HTTP_NOT_FOUND, 
$response->body());
}
return Apache2::Const::OK;
}
...

It's just for example, but you should get the point. $response is an object I 
return
from the functions doing the stuff. It encapsulates several aspects. I'm pretty
sure you can read that. $r is the well known mod_perl-object.

There is an issue with older IE-Browser which present the own message if the
message given to an error status is too short. You have to google for that.
As far as I can remember the solution is to extend the string. But I never 
bothered
about crappy browsers in error handling.

The line >>> if($response->body()) { <<< is interesting as it triggers the 
printing of
your own message or - when not given - the error mechanism described by
Torsten. So you can have default error pages, e.g. nicely formatted pages for
customers saying something meaningful like 'A error has occurred'. Or you can
feed informations for certain cornercases or debugging.

Hope I could help. Greetings to Torsten.

Best regards
Andreas Mock


> -Original Message-
> From: David Hofmann [mailto:elmic11...@gmail.com]
> Sent: Tuesday, October 25, 2011 5:20 PM
> To: modperl@perl.apache.org
> Subject: Mod Perl - Custom Error Pages
> 
> For a while now my system has error trapped issues the it encounters
> and printed an error page with a 200 status code.
> 
> Recently when my web developer started using AJAX, this became a
> problem, be cause he couldn't check for error codes on the pages. So I
> updated my error routine to pass proper error code like:
> 
> print "Content-type: text/html\n";
> print "Status: 400 Bad Request", "\n\n";
> 
> The problem I am however encountering is my custom error page
> generated by the perl script is displayed and then bellow it on the
> same page is the error page defined in the apache conf for that code.
> 
> The reason I want the mod perl to generate the error page is I want to
> be able to pass error info in the html comments.
> 
> Is there a way to tell get mod perl to tell apache to not send the
> default page also?
> 
> David

AW: Coupling Plack/PSGI with mod_perl, or an alternate architecture?

[Andreas Mock]

Hi Tatsuhiko,

which proxy do you use in front?

Best regards
McA


-Ursprüngliche Nachricht-
Von: Tatsuhiko Miyagawa [mailto:miyag...@gmail.com] 
Gesendet: Sonntag, 14. Oktober 2012 20:57
An: modperl@perl.apache.org
Betreff: Re: Coupling Plack/PSGI with mod_perl, or an alternate
architecture?

On Fri, Oct 12, 2012 at 11:59 AM, Fred Moyer  wrote:
>
> - A stable history of distribution packaging. Odds are the platform 
> you are using has mod_perl / Apache rpms/deb/ebuilds/etc for quite 
> some time.

These days more developers want to build (and even deploy) their own perl
using tools such as perlbrew, not the one that comes with the distribution,
and building perl _and_ mod_perl is certainly doable as mentioned elsewhere
in this thread, but is a separate step that's not required in most PSGI
perl-based server setup.

> - Knowledge that mod_perl has been proven in large scale deployments.
> The biggest I've worked on is about 500 servers if I recall correctly.

PSGI based deployment has been used and deployed in many online services,
small to large, personal to enterprise to PaaS, such as:
DeNA/ngmoco, livedoor, NHN, BBC, mixi, Booking, Stackato, dotcloud, just to
name a few (that I know).

Also: search.cpan.org and metacpan.org both run on top of PSGI (Starman and
Twiggy), so does cpanmetadb.plackperl.org, the backend of cpanminus -
although it serves about millions of requests per day, it isn't considered a
large scale deployment since it runs as one process on one server :)



--
Tatsuhiko Miyagawa

AW: [OT] AW: Unsuccessful stat on filename containing newline in RegistryCooker.pm

[Andreas Mock]

Hi all,

please, please don't forget to restrict yourself
to the 8.3 naming convention. Anything else is
troublesome... 
... the executable perl.exe is not named by
accident but follows deep wisdom.

;-)

McA

-Ursprüngliche Nachricht-
Von: demerphq [mailto:demer...@gmail.com] 
Gesendet: Donnerstag, 28. Februar 2013 15:14
An: mod_perl list
Betreff: Re: [OT] AW: Unsuccessful stat on filename containing newline in
RegistryCooker.pm

On 28 February 2013 13:29, André Warnier  wrote:
> alexander.elg...@external.t-systems.com wrote:
> ...
>
>> I am pretty shure, I tested the capability of filenames on a Linux (ext2)
>> or Solaris (ufs) filesystem, a long time ago
>> The outcome was, that you can use 254 different characters, except the
>> '\0' and the '/'.
>>
>> But I agree, it is an annoyance it should be forbidden. ;)
>>
>
> The person who invented that spaces and other unprintable characters were
> allowed in filenames and paths should be found, his PC and iPhone should
be
> confiscated, he should be exiled to an isolated island in the middle of
the
> Arctic Ocean, and he should be nominated for an IgNobel.  The person at
> Microsoft who decided that "C:\Program Files" was an acceptable place to
> install programs should be sent along as his butler.

What a stupid thing to say.

You can use those things on pretty much every file system and OS. Ever
since your grandmother started using a computer and thought it was
pretty darn stupid for young whippersnappers like yourself to tell her
that she couldnt name a file "good recipies".

A special place in hell is reserved for programmers that write code
that assumes that spaces and other unprintables are illegal in a
filename.

Yves







-- 
perl -Mre=debug -e "/just|another|perl|hacker/"

AW: Custom response problem

[Andreas Mock]

Have you loaded Apache2::RequestRec?

 

Best regards

Andreas Mock

 

 

Von: John Dunlap [mailto:j...@lariat.co] 
Gesendet: Dienstag, 18. März 2014 16:59
An: mod_perl list
Betreff: Custom response problem

 

I recently upgraded my workstation from Debian 6 to Debian 7 and I'm now
encountering a problem that I haven't seen before. My apache version is
2.2.22-13+deb7u1. My mod_perl version is 2.0.7-3. I'm guessing that I have
an installation problem of some kind but I'm not sure where to look for
problems. My application works correctly until I attempt to define a custom
response, as follows,

 

sub handler {

my $apache = shift; 

 
$apache->custom_response(Apache2::Const::HTTP_INTERNAL_SERVER_ERROR, 'hi
mom');

return Apache2::Const::HTTP_INTERNAL_SERVER_ERROR;

}

 

I see a 500 error, which is what I want, when I access the page. However the
error page is the default apache 500 error response page and I want to
override it. When I look in the logs, I see this:

 

[Tue Mar 18 15:41:32 2014] [error] [client 127.0.0.1] Can't locate object
method "custom_response" via package "Apache2::RequestRec" at
/usr/local/lariat-trunk/qa-trunk/lib/Lariat/V4/WS/RS/BootstrapHandler.pm
line 41.

 

This would imply, at least to me, that this method is not compiled into
mod_perl or perhaps into apache itself but I cannot be sure. Any
suggestions?

 

Cheers!

John

 

AW: Custom response problem

[Andreas Mock]

What does this give to you?

 

perl -MApache2::RequestRec -e'print Apache2::RequestRec->VERSION'

 

 

Regards

Andreas

 

 

Von: John Dunlap [mailto:j...@lariat.co] 
Gesendet: Dienstag, 18. März 2014 17:16
An: Andreas Mock
Cc: mod_perl list
Betreff: Re: Custom response problem

 

I've tried it with "use Apache2::RequestRec;" at the top of my handler and
without it. The outcome is the same in both cases. If I attempt to install
it from CPAN, it says that it is already installed.

 

On Tue, Mar 18, 2014 at 12:13 PM, Andreas Mock  wrote:

Have you loaded Apache2::RequestRec?

 

Best regards

Andreas Mock

 

 

Von: John Dunlap [mailto:j...@lariat.co] 
Gesendet: Dienstag, 18. März 2014 16:59
An: mod_perl list
Betreff: Custom response problem

 

I recently upgraded my workstation from Debian 6 to Debian 7 and I'm now
encountering a problem that I haven't seen before. My apache version is
2.2.22-13+deb7u1. My mod_perl version is 2.0.7-3. I'm guessing that I have
an installation problem of some kind but I'm not sure where to look for
problems. My application works correctly until I attempt to define a custom
response, as follows,

 

sub handler {

my $apache = shift; 

 
$apache->custom_response(Apache2::Const::HTTP_INTERNAL_SERVER_ERROR, 'hi
mom');

return Apache2::Const::HTTP_INTERNAL_SERVER_ERROR;

}

 

I see a 500 error, which is what I want, when I access the page. However the
error page is the default apache 500 error response page and I want to
override it. When I look in the logs, I see this:

 

[Tue Mar 18 15:41:32 2014] [error] [client 127.0.0.1] Can't locate object
method "custom_response" via package "Apache2::RequestRec" at
/usr/local/lariat-trunk/qa-trunk/lib/Lariat/V4/WS/RS/BootstrapHandler.pm
line 41.

 

This would imply, at least to me, that this method is not compiled into
mod_perl or perhaps into apache itself but I cannot be sure. Any
suggestions?

 

Cheers!

John

 

 

AW: Custom response problem

[Andreas Mock]

Thia is what I meant with "loading".

So, I should have been more precise: "using"  :-)

 

Andreas

 

 

Von: James Smith [mailto:j...@sanger.ac.uk] 
Gesendet: Dienstag, 18. März 2014 17:25
An: modperl@perl.apache.org
Betreff: Re: Custom response problem

 


Try:

 use Apache2::Response ();

This should add the method to $r (a lot of the Apache2:: modules do this -
Apache2::RequestUtil, Apache2::Upload etc)
On 18/03/2014 16:16, John Dunlap wrote: 

I've tried it with "use Apache2::RequestRec;" at the top of my handler and
without it. The outcome is the same in both cases. If I attempt to install
it from CPAN, it says that it is already installed.

 

On Tue, Mar 18, 2014 at 12:13 PM, Andreas Mock  wrote:

Have you loaded Apache2::RequestRec?

 

Best regards

Andreas Mock

 

 

Von: John Dunlap [mailto:j...@lariat.co] 
Gesendet: Dienstag, 18. März 2014 16:59
An: mod_perl list
Betreff: Custom response problem

 

I recently upgraded my workstation from Debian 6 to Debian 7 and I'm now
encountering a problem that I haven't seen before. My apache version is
2.2.22-13+deb7u1. My mod_perl version is 2.0.7-3. I'm guessing that I have
an installation problem of some kind but I'm not sure where to look for
problems. My application works correctly until I attempt to define a custom
response, as follows,

 

sub handler {

my $apache = shift; 

 
$apache->custom_response(Apache2::Const::HTTP_INTERNAL_SERVER_ERROR, 'hi
mom');

return Apache2::Const::HTTP_INTERNAL_SERVER_ERROR;

}

 

I see a 500 error, which is what I want, when I access the page. However the
error page is the default apache 500 error response page and I want to
override it. When I look in the logs, I see this:

 

[Tue Mar 18 15:41:32 2014] [error] [client 127.0.0.1] Can't locate object
method "custom_response" via package "Apache2::RequestRec" at
/usr/local/lariat-trunk/qa-trunk/lib/Lariat/V4/WS/RS/BootstrapHandler.pm
line 41.

 

This would imply, at least to me, that this method is not compiled into
mod_perl or perhaps into apache itself but I cannot be sure. Any
suggestions?

 

Cheers!

John

 

 



-- The Wellcome Trust Sanger Institute is operated by Genome Research
Limited, a charity registered in England with number 1021457 and a company
registered in England with number 2742969, whose registered office is 215
Euston Road, London, NW1 2BE. 

Apache2, mod_perl, getting the raw request ody

[Andreas Mock]

Hi all,

searched around and didn't find an answer. Probably it's complicated or
so simple that I don't see it.

I'm using mod_perl with the response handler being ModPerl::Registry.

I wrapped the call to the function handling the request in an
eval block to react on exceptions.

Is there a easy way to get (and dump) the raw request body?
I digged around the Apache2::-modules but didn't find something
appropriate.

Best regards
Andreas Mock

AW: Weird issues using DBI + mod_perl

[Andreas Mock]

Hi Bruce,

I'm just guessing, but this sounds much like reusing or double using of Oracle 
db handles, context handles or statement handles.

You have to have a look at the bookkeeping of the handles per request. Also 
forking of the childs may byte you.
You must ensure that db handles are ONLY opened in the child process doing the 
work and get closed properly.

With DBI in use you can use the variable DBI_TRACE to get extensive log. Google 
for that in combination with mod_perl.

Just some hints.

Best regards
Andreas


-Ursprüngliche Nachricht-
Von: Bruce Johnson  
Gesendet: Freitag, 16. August 2019 23:36
An: dbi users ; mod_perl list 
Betreff: Weird issues using DBI + mod_perl

So I’ve built a site using mod_perl in CGI mode, with DBI and Oracle as my 
database back end.

Everything works fine for a while but then I get intermittent weird errors that 
start happening for every page load, which is resulting in non-functional pages 
and error messages “Something happened contact your system administrator” 

"DBD::Oracle::st execute failed: ORA-01008: not all variables bound (DBD ERROR: 
OCIStmtExecute) [for Statement "select to_char(add_months(sysdate, 24), 
'MM/DD/'), to_char(add_months(sysdate, 24), 'J') from dual”]” at 
/home/allwebfiles/perl/LocalModules/Cal4Defaults.pm line 267,  line 522.\n

There are no bound variables in the query...

Another one:  [error] DBD::Oracle::st execute failed: ORA-01007: variable not 
in select list (DBD ERROR: OCIStmtExecute) [for Statement "select 
to_char(add_months(sysdate, 24), 'MM/DD/'), to_char(add_months(sysdate, 
24), 'J') from dual"] at /home/allwebfiles/perl/LocalModules/Cal4Defaults.pm 
line 267,  line 522.\n

Other errors that happen are :

DBD::Oracle::st execute failed: ORA-01007: variable not in select list (DBD 
ERROR: OCIStmtExecute) [for Statement "select pid, cn, email, afflist, bldg_id, 
start_task from login_info where cookie =?" with ParamValues: 
:p1='NS9DdciuH7XPeSVygRyUjoviZ’] 

DBD::Oracle::db prepare failed: ORA-01002: fetch out of sequence (DBD ERROR: 
OCIStmtExecute/Describe) [for Statement "select to_char(add_months(sysdate, 
24), 'MM/DD/'), to_char(add_months(sysdate, 24), 'J') from dual"] at 
/home/allwebfiles/perl/LocalModules/Cal4Defaults.pm line 266,  line 522.\n

DBD::Oracle::st execute failed: ORA-03106: fatal two-task communication 
protocol error (DBD ERROR: OCIStmtExecute) [for Statement "select pid, cn, 
email, afflist, bldg_id, start_task from login_info where cookie =?" with 
ParamValues: :p1='v2bpY8jnoPw5yf0x71I4wZPx6'] at 
/home/allwebfiles/perl/LocalModules/Cal4Defaults.pm line 307

Most of these errors (that cause the DBI stuff to break) are happening in this 
defaults module, which IS referenced more than once per page load as the pages 
have ajax calls to other scripts that reference this module.

restarting httpd fixes the problem, for a while. This isn’t a very heavily 
loaded site, right now it’s the middle of the afternoon and I'm seeing 
'accesses per minute' kind of traffic.

And clues as to where I should start to look?  

-- 
Bruce Johnson
University of Arizona
College of Pharmacy
Information Technology Group

Institutions do not have opinions, merely customs

AW: Weird issues using DBI + mod_perl

[Andreas Mock]

Hi Bruce,

 

thank you for the feedback.

 

Have you posted it under "#use" ? :-

 

Best regards

Andreas

 

 

Von: Bruce Johnson  
Gesendet: Dienstag, 3. September 2019 18:36
An: Andreas Mock 
Cc: dbi users ; mod_perl list 
Betreff: Re: Weird issues using DBI + mod_perl

 

Just a follow up…it was embarrassingly stupid. You know how you can stare at an 
error in a line dozens of times and not see it? Like the ‘#' commenting out 
"use Apache::DBI” in your startup.pl file ... 





On Aug 16, 2019, at 2:47 PM, Andreas Mock mailto:andreas.m...@web.de> > wrote:

 

Hi Bruce,

I'm just guessing, but this sounds much like reusing or double using of Oracle 
db handles, context handles or statement handles.

You have to have a look at the bookkeeping of the handles per request. Also 
forking of the childs may byte you.
You must ensure that db handles are ONLY opened in the child process doing the 
work and get closed properly.

With DBI in use you can use the variable DBI_TRACE to get extensive log. Google 
for that in combination with mod_perl.

Just some hints.

Best regards
Andreas

 

-- 
Bruce Johnson
University of Arizona
College of Pharmacy
Information Technology Group

Institutions do not have opinions, merely customs

 

AW: STDOUT Problem migrating a site to mod_perl

[Andreas Mock]

Hi Thomas,

why not implementing the follwoing steps.

1) create tmp file.
2) convert to tmp file
3) if everything worked fine reading file, setting appropriate headers and 
print content and afterwards delete tmp file.
4) if something went wrong you can send an error page with appropriate headers 
and code.

I would avoid messing around with STDOUT.

Best regards
Andreas



> -Ursprüngliche Nachricht-
> Von: Thomas Kenyon 
> Gesendet: Montag, 23. September 2019 14:52
> An: modperl@perl.apache.org
> Betreff: STDOUT Problem migrating a site to mod_perl
> 
> I have run into an odd problem when migrating a site to use mod_perl.
> 
> The script in question, builds some postscript into a variable, then
> uses psconvert (from Postscript::Convert) to change that into a png and
> sends that back to the webserver. (headers are set up earlier in the
> script).
> 
> This seems to work well with mod_cgi but doesn't work with mod_perl
> using either ModPerl::Registry or ModPerl::PerlRun.
> 
> Annoyingly psconvert doesn't return the converted image, instead it
> dumps it to a file (or stdout).
> 
> I have tried to re-implement this using GSAPI, but GSAPI doesn't seem to
> build on any of my machines. (all debian 9/10 based).
> 
>  use PostScript::Convert;
>  psconvert(\$ps, '%stdout%' , format => 'png', resolution => '72',
> paper_size => '6.5x4');
> 
> I've searched through as much documentation as I can find, and tried
> various things, but none seem to work.
> 
> TIA for any guidance.

AW: Setting a MIME type on $r->custom_response

[Andreas Mock]

Hi Andrew,

 

have a look at 

 

$r->err_headers_out->add('Content-Type' => 'bla bla');

 

Best regards

Andreas

 

 

Von: Andrew Green  
Gesendet: Dienstag, 24. September 2019 18:39
An: modperl@perl.apache.org
Betreff: Setting a MIME type on $r->custom_response

 

Hi all,

Is it possible to get $r->custom_response to respond using a MIME type other 
than text/html?


I’m writing a mod_perl application that’s intended to work with someone else’s 
React front end.  My code therefore just sends JSON.

 

For an authentication failure, I can use custom_response to send a custom JSON 
error with a Forbidden HTTP status:

 

$r->custom_response(Apache2::Const::FORBIDDEN, $json);

 

It works fine, except that I can’t get it to use the correct MIME type.  The 
following has no effect (whether I deploy it immediately before the 
custom_response call or afterwards):

 

$r->content_type("application/json; charset=utf-8”);

 

It’s not a huge deal — and I haven’t yet tested whether React freaks out over 
the Forbidden HTTP status regardless.

 

But if there’s a way to do this, I’d love to know.  Thanks!

 

Cheers,

Andrew.


-- 
Andrew Green
Article Seven Limited
http://www.article7.co.uk/

Article Seven Limited is a registered company in England and Wales.  Registered 
number: 5703656. Registered office: 10 Hamilton Road, Sidcup, Kent, DA15 7HB.




 

AW: Question on a rewrite rule

[Andreas Mock]

You have to see in which virtualhost section this rule was put into.
I'm pretty sure it's only in the http section.

-Ursprüngliche Nachricht-
Von: Wesley Peng  
Gesendet: Donnerstag, 9. Januar 2020 03:44
An: modperl@perl.apache.org
Betreff: Question on a rewrite rule

Hello

After installation of Letsencrypt free SSL, it adds a rule into apache's config 
file:

RewriteCond %{SERVER_NAME} =sample.com
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]


I know it means when SERVER_NAME is sample.com, it rewrites the plain HTTP 
request to HTTPS request.

My question is, is this not a dead loop?

For HTTPS request, the SERVER_NAME is also matched, so it rewrites again and 
again...

My guess it's a dead rewrite loop, but it actually works fine.

I am not sure about this, please help.

Thanks.

AW: Can't locate object method "new" via package "CGI::Cookie"

[Andreas Mock]

Hi Vincent,

I don't have Debian. Therefor just a view hints.

1) CGI::Cookie loading is done sometimes dynamically via require. E.g. in
CGI.pm itself.
2) mod_perl 1/2 with the helper modules Apache2::RequestUtil interact
dynamically.
3) It can be possible that you have a version clash (alone or in combination
with other modules). Probably there was once pulled a module in your older
server which hinders the same behaviour.

Tests:
perl -MCGI::Cookie -E'say CGI::Cookie->VERSION'
perl -MApache2::RequestUtil -E'say Apache2::RequestUtil->VERSION'

On the other hand you can add debug code just before line
/home/lib/Marica/Base/login.pm line 198
to inspect the %INC hash to see whether CGI::Cookie was loaded somewhere.

You can also find the file Cookie.pm in your include path and put a
print STDERR map { "$_\n" } caller;
into it to find the location in Apache error log where it gets loaded in
the module stack.

Best regards
Andreas


-Ursprüngliche Nachricht-
Von: Vincent Veyron  
Gesendet: Sonntag, 31. Mai 2020 20:50
An: modperl@perl.apache.org
Betreff: Can't locate object method "new" via package "CGI::Cookie"

Hi List,

I have two servers using mod_perl, both having the same modules installed
and enabled; server1 got a fresh install of Debian Buster, while server2 was
upgraded to Buster from Stretch.

This code :
my $cookie = CGI::Cookie->new(-name  => 'session',
  -value => $session_id,
  -domain => $hostname
) ;

generates the following error on server2 :

[Sun May 31 19:54:03.053725 2020] [perl:error] [pid 30663] [client
83.113.48.133:59196] Can't locate object method "new" via package
"CGI::Cookie" (perhaps you forgot to load "CGI::Cookie"?) at
/home/lib/Marica/Base/login.pm line 198.\n, referer:
https://marica.fr/site/abonnes/index.html

It works fine if add "use CGI::Cookie" to my module. 

What could be the reason why it works on server1 without "use CGI::Cookie"
and not on server2?

-- 
https://compta.libremen.com
Logiciel libre de comptabilité générale en partie double

AW: suggestions for perl as web development language [EXT]

[Andreas Mock]

Hi all,

we also have a big code base whose starting point is more than two decades ago.
If you have developers who know their stuff and all the internal developed 
modules
and helpers you're good to go. BUT: We do have problems to get young, fresh
perl developers. Why? This language is simply unattractive to young people.

If I had to start a web application today on an empty field I wouldn't choose 
perl. 

There is another point. When we started all logic was done in the backend.
Nowadays much is done in the browser itself with Javascript. You need this
Javascript knowhow in any case. And when you have it there is no big step
to using this in the backend too.

So, in my opinion there is a relevant difference between starting a project
or maintaining an (very) old one. Maintaining an old code base where you're
stuck to years of old code is not attractive to new developers in any case.

mod_perl? The opinions may vary. But we're trying to get rid of it. In the
old days mod_perl with apache was the one and only process doing
everything. Now it's common to have serveral tiers. In the front a
load balancer which probably also terminates SSL. Some lightweight
servers serving static stuff as fast as possible, and one or more servers which
act as application server. As soon as several stages of request processing
are not done in Apache (e.g. Rewriting, Dispatching, Logging) you only
use one stage of Apache to produce the content. And this can be done
with a native perl application server.

Don't understand me wrong. We program in Perl, we know Perl more or less,
we have a big code base in Perl. But I won't start a new project on an empty (!)
field with it. But our field is NOT empty because we have so many known own
libraries and modules for all the common cross application requirements that
it would last very long to build these in a different language.

At the end you need people to do it. And you have to see how much legacy 
code base and knowledge is there. 

Regards
Andreas

-Ursprüngliche Nachricht-
Von: James Smith  
Gesendet: Dienstag, 4. August 2020 09:37
An: Wesley Peng ; modperl@perl.apache.org
Betreff: RE: suggestions for perl as web development language [EXT]

Perl is a great solution for web development.

Others will disagree but the best way I still believe is using mod_perl - but 
only if you use it's full power - and you probably need a special sort of mind 
set to use - but that can be said for any language.

>From experience - it may be fractionally slower than small "standalone" apps 
>that dancer etc are good at, but it is (a) much, much more stable {dancer etc 
>does not cope well with either large requests or lots of small requests}, and 
>(b) if you have a large code base and/or a large number of services then it 
>generally uses much less compute power than the others {can easily handle 
>multiple services on a single apache instance}

Where it really gains is the hooks into the apache process - being able to add 
functionality easily at any stage in the request process, from path 
translation, AAA stages, pre-processing, to post-processing and logging, and 
also to interact with other languages at any stage - e.g. can handle 
pre-processing & post-processing around a script written in another language 
(e.g. PHP, Java) or produced by another webserver integrated by mod_proxy.

It isn't really a framework though like dancer or mojolicious and thus has its 
own advantages and disadvantages.

You would to some extent have to roll your own code to produce the pages 
themselves although there are libraries out there to do lots of it.

We have an in house library whose embryonic stages were written over 20 years 
ago - and has now been stable for around 12-13 years and works strong...

James

-Original Message-
From: Wesley Peng 
Sent: 04 August 2020 06:43
To: modperl@perl.apache.org
Subject: suggestions for perl as web development language [EXT]

greetings,

My team use all of perl, ruby, python for scripting stuff.
perl is stronger for system admin tasks, and data analysis etc.
But for web development, it seems to be not as popular as others.
It has less selective frameworks, and even we can't get the right people to do 
the webdev job with perl.
Do you think in today we will give up perl/modperl as web development language, 
and choose the alternatives instead?

Thanks & Regards




--
 The Wellcome Sanger Institute is operated by Genome Research  Limited, a 
charity registered in England with number 1021457 and a  company registered in 
England with number 2742969, whose registered  office is 215 Euston Road, 
London, NW1 2BE.

Re: Apache server returns early before process is complete

[Andreas Mock]

Hi Joseph,

your description is very vague, so can only answer on some assumptions:

It sounds like a timeout is fired somewhere.

Best advice in these situations: Log as many steps as you can. Keep your 
eyes open on TCP/IP and higher level timeouts.


Declare only ONE instance responsible for a retry: Either the app server 
calling the dispatcher with several tries or the dispatcher trying for 
himself. Not both.


Best regards
Andreas


Am 23.04.2025 um 16:21 schrieb Joseph He:

All, good day.

Here is the issue I have.
My entire application is running on ModPerl/Apache environment.
I send Http::Request with data load from my App server to a dispatch 
server thru LWP::UserAgent, I set the timeout 600 seconds.


The dispatch server is supposed to manipulate the data and send the 
data to an external SFTP server. Because the SFTP can fail, it will 
keep trying up to 4 times with 30 seconds sleep in case that SFTP 
connection fails.


Recently, I found that I uploaded the file twice sometimes. I figured 
out the root cause is that my Dispatch server returns 'failure' at 6 
minutes while it keeps trying to do the SFTP. The App server 
received HTTP::Response with error status so it issued another call to 
send data. It turns out I uploaded the identified file twice.


Anybody has this sort of experience? Why does the dispatch server 
return 'error' while it still processes the data?


Thanks a lot,
Joseph

Re: Apache server returns early before process is complete

[Andreas Mock]

Hallo Joe,

try to reduce the problem.

Make the call to your SFTP-Service via curl or some other http(s) client 
to see whether you get the same timeout. If yes, than the server side is 
closing the connection. If not then you have to investigate the 
LWP::UserAgent part.


Another hint in combination with SSL: 
https://stackoverflow.com/questions/9400068/make-timeout-work-for-lwpuseragent-https


Best regards
Andreas


Am 13.05.2025 um 17:22 schrieb Joseph He:

Andreas, thank you.

On the client side, I set the timeout at LWP::UserAgent request to 
600, and I can verify that it indeed works on my QA and DEV 
environment. If I change it to 120, then it can timeout at 120.
So on my production server, the client side receives a timeout from 
the server after 5 minutes, so I still think the server Timeout plays 
a role here. I just don't know what config I can change to test it out.


Joe

On Tue, May 13, 2025 at 10:07 AM Andreas Mock  wrote:

Hi Joe,

when you send a request via LWP::UserAgent to the Server which
does the long lasting SFTP calls, then I'm pretty sure that you
get a timout in the LWP::UserAgent code.

I'm pretty sure the client (LWP::UserAgent) is not waiting long
enough for the answer: https://metacpan.org/pod/LWP::UserAgent#timeout

After having here a long timeout you have to be sure that the very
first client which sent the very first request also waits long
enough to let the application server make severals tries,
therefore n * timeout.

Best wishes
Andreas


Am 13.05.2025 um 16:46 schrieb Joseph He:

Many thanks to you all.

I am still trying to figure out the issue. Let me re-explain the
problem I experienced with some details.

The environment is Ubuntu 22.04, Apache2, ModPerl.
I run a Http::request with LWP::UserAgent, the server receives
the request and starts to process it.
But it takes much longer due to a stalled SFTP call to the remote
server, the Apache server timeout and sends back failure,
meanwhile,*the server actually is still trying to process this
request*.
On the calling side, after receiving the failure status, it
initiates another http::request and the load balancer redirects
this call to another server for processing.
It turns out this same http::request is processed twice.

On my production server the timeout happens at 300 seconds mark.
On my QA and Dev server, the timeout happens at 600 seconds. I
have not changed anything on my production server yet.
But on my QA and DEV servers, I have tried to change Timeout in
apache2.conf, have tried to add Timeout to the virtualhost
config, also have tried to add SetPerlEnv MOD_PERL_TIMEOUT to the
virtualhost config, none of them change the timeout behavior of
my QA and DEV servers.

So what exactly controls the Timeout? I am totally lost.

Cheers,
Joe


On Wed, Apr 23, 2025 at 5:17 PM Mithun Bhattacharya
 wrote:

Okay agreed that is a valid time out basically it is saying
that a client has established tcp/ip connection but has not
put its request either a get put or a post

On Wed, Apr 23, 2025, 3:38 PM Joseph He
 wrote:

On Apache2 doc, I found this. How does this timeout work?
It looks like it can only wait for 300 seconds before
failing a request.

https://httpd.apache.org/docs/2.0/mod/core.html#timeout
Description:

<https://httpd.apache.org/docs/2.0/mod/directive-dict.html#Description>
Amount of time the server will wait for certain events
before failing a request
Syntax:
<https://httpd.apache.org/docs/2.0/mod/directive-dict.html#Syntax>
|TimeOut seconds|
Default:
<https://httpd.apache.org/docs/2.0/mod/directive-dict.html#Default>
|TimeOut 300|
Context:
<https://httpd.apache.org/docs/2.0/mod/directive-dict.html#Context>
server config, virtual host
Status:
<https://httpd.apache.org/docs/2.0/mod/directive-dict.html#Status>
Core
Module:
<https://httpd.apache.org/docs/2.0/mod/directive-dict.html#Module>
core

The |TimeOut| directive currently defines the amount of
time Apache will wait for three things:

 1. The total amount of time it takes to receive a GET
request.
 2. The amount of time between receipt of TCP packets on
a POST or PUT request.
 3. The amount of time between ACKs on transmissions of
TCP packets in responses.

We plan on making these separately configurable at some
point down the road. The timer used to default to 1200
bef

Re: Apache server returns early before process is complete

[Andreas Mock]

imeout is basically no response came
back. Why that happens is because we think we want
to have a correct response. Unfortunately for long
running requests the correct response shouldn't be
via http response code or we face situations like
this. Instead reply with a 200 OK immediately and
then provide correct status in the message body.
Once a response code/header has been sent timeout
won't trigger and you could potentially hold the
connection for hours without a problem.

On Wed, Apr 23, 2025, 9:32 AM Andreas Mock
 wrote:

Hi Joseph,

your description is very vague, so can only
answer on some assumptions:

It sounds like a timeout is fired somewhere.

Best advice in these situations: Log as many
steps as you can. Keep your
eyes open on TCP/IP and higher level timeouts.

Declare only ONE instance responsible for a
retry: Either the app server
calling the dispatcher with several tries or
the dispatcher trying for
himself. Not both.

Best regards
Andreas


Am 23.04.2025 um 16:21 schrieb Joseph He:
> All, good day.
>
> Here is the issue I have.
> My entire application is running on
ModPerl/Apache environment.
> I send Http::Request with data load from my
App server to a dispatch
> server thru LWP::UserAgent, I set the
timeout 600 seconds.
>
> The dispatch server is supposed to
manipulate the data and send the
> data to an external SFTP server. Because the
SFTP can fail, it will
> keep trying up to 4 times with 30 seconds
sleep in case that SFTP
> connection fails.
>
> Recently, I found that I uploaded the file
twice sometimes. I figured
> out the root cause is that my Dispatch
server returns 'failure' at 6
> minutes while it keeps trying to do the
SFTP. The App server
> received HTTP::Response with error status so
it issued another call to
> send data. It turns out I uploaded the
identified file twice.
>
> Anybody has this sort of experience? Why
does the dispatch server
> return 'error' while it still processes the
data?
>
> Thanks a lot,
> Joseph
>