Re: [mp2] No data is read from Perl Output Filter

[Vincent Veyron]

On Mon, 21 Nov 2016 16:23:22 -0500
"Vlad Liapko"  wrote:

> I am experimenting with ReverseProxy::FormFiller and stuck with output 
> request rewriting. Nothing is read from filter, but filter is invoked.
> Code below writes nothing into logs.
> while ($f->read(my $buffer, 1024)) {
>   $body .= $buffer; 
>($f->r,"-$body--$buffer-");
> }
> 
> There are always a few parameters are sent via POST during tests, but nothing 
> can't be rewritten by the filter because nothing is read.
> 

I think you need to use something like :

while ($f->read(my $buffer, 1024)) {
$body .= $buffer;
}
if ($f->seen_eos) {
($f->r,"-$body---");
}

You'll find relevant examples in:

http://perl.apache.org/docs/2.0/user/handlers/filters.html#C_PerlOutputFilterHandler_
http://modperlbook.org/html/25-3-I-O-Filtering.html

-- 
Bien à vous, Vincent Veyron 

https://marica.fr/
Gestion des sinistres assurances, des dossiers contentieux et des contrats pour 
le service juridique

[mp2] No data is read from Perl Output Filter

[Vlad Liapko]

I am experimenting with ReverseProxy::FormFiller and stuck with output request 
rewriting. Nothing is read from filter, but filter is invoked.
Code below writes nothing into logs.
while ($f->read(my $buffer, 1024)) {
  $body .= $buffer; 
   ($f->r,"-$body--$buffer-");
}

There are always a few parameters are sent via POST during tests, but nothing 
can't be rewritten by the filter because nothing is read.

Fwd: [mp2] No data is read from Perl Output Filter

[Vladimir L]

I am experimenting with ReverseProxy::FormFiller and stuck with output
request rewriting. Nothing is read from filter, but filter is invoked.
Code below writes nothing into logs.
while ($f->read(my $buffer, 1024)) {
  $body .= $buffer;
   ($f->r,"-$body--$buffer-");
}

There are always a few parameters are sent via POST during tests, but
nothing can't be rewritten by the filter because nothing is read.

mod_perl output filter and mod_proxy, mod_cache

[Tim Watts]

Hi,

Is it in theory possible to insert a perl output filter between 
mod_proxy and mod_cache?


Or at least between mod_proxy and the client?



The problem I'm trying to solve is this:

We have 100+ web servers where apache fronts a separate tomcat server 
using mod_proxy.


Sadly, the tomcat dev's forgot to set any caching headers in the HTTP 
response (either Expires, Last-Modified or Cache-control) so the sites 
are largely uncacheable by browsers and the various tomcats are becoming 
overloaded.


1/3 of our sites are typically invariant (the production sites have 
stable and unchanging data and most queries are via GET requests).


Therefore, the idea of forcing in some cache control headers en-route 
and also enabling some apache caching has a good chance of working well 
without affecting anything.


mod_headers and mod_proxy don't seem to play well together and mod-cache 
doesn't either (probably due to lack of cache control headers in the 
tomcat response, though I haven't proved this is actually the case).


So the thought of doing a perl based filter to insert cache-control 
headers occurred.


It is likely I can insert such a filter on Apache 2.2 *between* 
mod_proxy and mod_cache?


Or am I going to have to implement a filter that includes proxying 
and/or caching?


Many thanks for any advice,

Cheers,

Tim

--
Tim Watts
Personal Blog: http://www.dionic.net/tim/

Re: mod_perl output filter and mod_proxy, mod_cache

[André Warnier]

Tim Watts wrote:

Hi,

Is it in theory possible to insert a perl output filter between 
mod_proxy and mod_cache?


Or at least between mod_proxy and the client?



The problem I'm trying to solve is this:

We have 100+ web servers where apache fronts a separate tomcat server 
using mod_proxy.


Sadly, the tomcat dev's forgot to set any caching headers in the HTTP 
response (either Expires, Last-Modified or Cache-control) so the sites 
are largely uncacheable by browsers and the various tomcats are becoming 
overloaded.


1/3 of our sites are typically invariant (the production sites have 
stable and unchanging data and most queries are via GET requests).


Therefore, the idea of forcing in some cache control headers en-route 
and also enabling some apache caching has a good chance of working well 
without affecting anything.


mod_headers and mod_proxy don't seem to play well together and mod-cache 
doesn't either (probably due to lack of cache control headers in the 
tomcat response, though I haven't proved this is actually the case).


So the thought of doing a perl based filter to insert cache-control 
headers occurred.


It is likely I can insert such a filter on Apache 2.2 *between* 
mod_proxy and mod_cache?


Or am I going to have to implement a filter that includes proxying 
and/or caching?
 

(That would probably be difficult, inefficient or both)

Assuming that what you say about Tomcat is true (I don't know, and it may be worth asking 
this on the Tomcat list), I can think of another way to achieve what you seem to want :
if you can distinguish, from the request URL (or any other request property), the requests 
that are for invariant things, then you could arrange to /not/ proxy these requests to 
Tomcat, and serve them directly from Apache httpd.


Which proxying method exactly are you using between Apache and Tomcat ? (if you are using 
mod_proxy, then you are either using mod_proxy_http or mod_proxy_ajp; you could also 
consider using mod_jk).


Also, what are the versions of Apache and Tomcat that you are using ?

Re: mod_perl output filter and mod_proxy, mod_cache

[Tim Watts]

On 14/07/11 11:16, André Warnier wrote:

Hi Andre,

Thanks for the quick reply :)


(That would probably be difficult, inefficient or both)

Assuming that what you say about Tomcat is true (I don't know, and it
may be worth asking this on the Tomcat list), I can think of another way
to achieve what you seem to want :
if you can distinguish, from the request URL (or any other request
property), the requests that are for invariant things, then you could
arrange to /not/ proxy these requests to Tomcat, and serve them directly
from Apache httpd.


Indeed that is a good idea. We are doing that for new projects for css 
and js files (apache does not proxy certain paths and picks these up 
from the local filesystem).


We can't do that for the 100 odd legacy servers as no-one has time o 
delve into the java/JSP code. I need to do something outside of tomcat 
where possible. Just to explain, each web server is a paid-for project - 
and when it's done, it sits there for 5+ years.


Only I have the time/inclination to fix this as it's killing my VMWare 
infrastructure. Because the sites are all fronted by apache in a similar 
way, one solution is likely to apply to most of the sites.


I would also add that most of the sites are dynamically driven pages, 
even involving MySQL querying, but once launched, the data remains 
fairly static - eg GET X will always resolve to reponse Y.


I'm planning a small seminar on the value of Cache-Control for my dev 
colleagues so they can stop making this mistake ;- But that still 
leaves a lot of done projects to fix.



Which proxying method exactly are you using between Apache and Tomcat ?
(if you are using mod_proxy, then you are either using mod_proxy_http or
mod_proxy_ajp; you could also consider using mod_jk).


mod_proxy_http specifically.

mod_jk looks interesting for new projects (we have local tomcats for 
those now) - I think it may be a non-starter for old stuff as trying to 
retro fit it may not be so simple (our older tomcat servers are in a 
remote farm on their own machines hence the use of mod_proxy_http).



Also, what are the versions of Apache and Tomcat that you are using ?



Apache 2.2 (various sub versions) and both tomcat 5.5 and tomcat 6 (but 
all on remote machines listening on TCP sockets).


I think for this problem, I have to treat tomcat as a little, rather 
inefficient, black box and try to fixup on the apache front ends, hence 
the direction of my original idea...


Cheers,

Tim

--
Tim Watts
Personal Blog: http://www.dionic.net/tim/

Re: mod_perl output filter and mod_proxy, mod_cache

[Alex J. G. Burzyński]

Hi Tim,

If you are after caching the responses, maybe an easier solution would
be to use a reverse proxy - like Varnish?

You would be then in complete control over the incoming and outgoing
headers and could cache responses based on the url / inject Expires
headers so browsers could cache them too etc.

Cheers,
Alex


On 14/07/11 11:39, Tim Watts wrote:
 On 14/07/11 11:16, André Warnier wrote:

 Hi Andre,

 Thanks for the quick reply :)

 (That would probably be difficult, inefficient or both)

 Assuming that what you say about Tomcat is true (I don't know, and it
 may be worth asking this on the Tomcat list), I can think of another way
 to achieve what you seem to want :
 if you can distinguish, from the request URL (or any other request
 property), the requests that are for invariant things, then you could
 arrange to /not/ proxy these requests to Tomcat, and serve them directly
 from Apache httpd.

 Indeed that is a good idea. We are doing that for new projects for css
 and js files (apache does not proxy certain paths and picks these up
 from the local filesystem).

 We can't do that for the 100 odd legacy servers as no-one has time o
 delve into the java/JSP code. I need to do something outside of
 tomcat where possible. Just to explain, each web server is a paid-for
 project - and when it's done, it sits there for 5+ years.

 Only I have the time/inclination to fix this as it's killing my VMWare
 infrastructure. Because the sites are all fronted by apache in a
 similar way, one solution is likely to apply to most of the sites.

 I would also add that most of the sites are dynamically driven
 pages, even involving MySQL querying, but once launched, the data
 remains fairly static - eg GET X will always resolve to reponse Y.

 I'm planning a small seminar on the value of Cache-Control for my dev
 colleagues so they can stop making this mistake ;- But that still
 leaves a lot of done projects to fix.

 Which proxying method exactly are you using between Apache and Tomcat ?
 (if you are using mod_proxy, then you are either using mod_proxy_http or
 mod_proxy_ajp; you could also consider using mod_jk).

 mod_proxy_http specifically.

 mod_jk looks interesting for new projects (we have local tomcats for
 those now) - I think it may be a non-starter for old stuff as trying
 to retro fit it may not be so simple (our older tomcat servers are in
 a remote farm on their own machines hence the use of mod_proxy_http).

 Also, what are the versions of Apache and Tomcat that you are using ?


 Apache 2.2 (various sub versions) and both tomcat 5.5 and tomcat 6
 (but all on remote machines listening on TCP sockets).

 I think for this problem, I have to treat tomcat as a little, rather
 inefficient, black box and try to fixup on the apache front ends,
 hence the direction of my original idea...

 Cheers,

 Tim

Re: mod_perl output filter and mod_proxy, mod_cache

[Tim Watts]

On 14/07/11 11:52, Alex J. G. Burzyński wrote:

Hi Tim,

If you are after caching the responses, maybe an easier solution would
be to use a reverse proxy - like Varnish?

You would be then in complete control over the incoming and outgoing
headers and could cache responses based on the url / inject Expires
headers so browsers could cache them too etc.

Cheers,
Alex



[Sorry Alex, hit reply instead of reply-list]

Hi Alex,

I was initially also thinking Squid - but it's rather heavy.

I have not come across Varnish but having a quick look (and noting it is 
available on Debian - good) it looks like a damn good option.


I think you are right - apache is great, but the order of execution of 
modules is not well documented and prone to changing (hence my original 
question here) and trying to splice effectively 3 filters together 
(proxy, header-fiddling and cache) is probably doomed to grief.


Thanks for the tip - I'm off to try that today!

All the best,

Tim

--
Tim Watts
Personal Blog: http://www.dionic.net/tim/

Re: mod_perl output filter and mod_proxy, mod_cache

[André Warnier]

Hi.

I have to apologise.
I misunderstood your first post, and I wanted to verify on the Tomcat list, so I quoted 
the following passage of your first post in my message there :


Sadly, the tomcat dev's forgot to set any caching headers in the HTTP response (either 
Expires, Last-Modified or Cache-control) so the sites are largely uncacheable by browsers 
and the various tomcats are becoming overloaded.


Unfortunately, the Tomcat Dev's there took it rather seriously, and as a consequence now 
you name is shit on the Tomcat list.



.. just kidding, I did not quote your name.

Anyway, apart from a few huffed responses to my misquote (since then rectified), someone 
provided a suggestion that may not be the simplest, but might be helpful anyway in some 
cases :


Have a look at : http://www.tuckey.org/urlrewrite/

This is a Java Servlet Filter, which can be added transparently around any Tomcat web 
application (by adding the required section in the web.xml config file of that web 
application).
Java Servlet Filters are such that the Tomcat web application is not even aware that it is 
there, and continues to work as before.  Much like Apache input and output filters in 
fact, except that a Java Servlet Filter is both at the same time (it wraps the webapp on 
both sides).


Anyway, this filter can do such things as conditionally or not adding response headers to 
anything the webapp produces.  And it can do much more, as with time it has evolved into 
some kind of mish-mash of mod_rewrite, mod_headers and mod_proxy.


It is more one-by-one work than doing something at the Apache front-end level or via a 
proxy, but it also provides better fine-tuning possibilities.

So, if you can for instance easily identify the worst offenders, it might be an 
option.

And it is certainly a good tool to have in one's toolcase.

Re: mod_perl output filter and mod_proxy, mod_cache

[Tim Watts]

On 14/07/11 12:43, André Warnier wrote:

Hi.

I have to apologise.
I misunderstood your first post, and I wanted to verify on the Tomcat
list, so I quoted the following passage of your first post in my message
there :

Sadly, the tomcat dev's forgot to set any caching headers in the HTTP
response (either Expires, Last-Modified or Cache-control) so the sites
are largely uncacheable by browsers and the various tomcats are becoming
overloaded.

Unfortunately, the Tomcat Dev's there took it rather seriously, and as a
consequence now you name is shit on the Tomcat list.


.. just kidding, I did not quote your name.


LoL - I hate tomcat anyway (for it's fatness) so I don't mind if they 
hate me ;-


I should have clarified as my Department's dev team (ie the ones who 
use tomcat here) rather than the Tomcat Developers themselves...


I have no doubts that jsp can be told to emit certain headers but for 
some reason a lot of web developers IME often miss the finer points of 
HTTP. This of course would be the correct place to do it as they can 
choose different max-age times to suit the content.


I plan to run a 20 minute seminar on this specific point for my lot (and 
more such seminars for other issues like security and SQL efficiency) 
but that still leaves loads of old black-boxes to manage for a few years.



Anyway, apart from a few huffed responses to my misquote (since then
rectified), someone provided a suggestion that may not be the simplest,
but might be helpful anyway in some cases :

Have a look at : http://www.tuckey.org/urlrewrite/

This is a Java Servlet Filter, which can be added transparently
around any Tomcat web application (by adding the required section in
the web.xml config file of that web application).
Java Servlet Filters are such that the Tomcat web application is not
even aware that it is there, and continues to work as before. Much like
Apache input and output filters in fact, except that a Java Servlet
Filter is both at the same time (it wraps the webapp on both sides).


That could be interesting too - as long as it's something I can bolt in 
without having to recompile the webapp code, I'm game. As a linux 
sysadmin, I draw a clear line between the systems (my problem) and the 
apps (dev team) - and not knowing java (much) I'm not qualified to mess 
with their stuff... I'm happy to go as far as messing with server.xml 
and web.xml though :)



Anyway, this filter can do such things as conditionally or not adding
response headers to anything the webapp produces. And it can do much
more, as with time it has evolved into some kind of mish-mash of
mod_rewrite, mod_headers and mod_proxy.

It is more one-by-one work than doing something at the Apache front-end
level or via a proxy, but it also provides better fine-tuning
possibilities.
So, if you can for instance easily identify the worst offenders, it
might be an option.

And it is certainly a good tool to have in one's toolcase.


I agree - I'll have a look at that after I play with Alex's suggestion 
of Varnish :)


Thanks very much for your time :)

all the best,

Tim

--
Tim Watts
Personal Blog: http://www.dionic.net/tim/

Re: mod_perl output filter and mod_proxy, mod_cache

[James Smith]

On 14/07/2011 11:39, Tim Watts wrote:

On 14/07/11 11:16, André Warnier wrote:

Hi Andre,

Thanks for the quick reply :)


(That would probably be difficult, inefficient or both)

Assuming that what you say about Tomcat is true (I don't know, and it
may be worth asking this on the Tomcat list), I can think of another way
to achieve what you seem to want :
if you can distinguish, from the request URL (or any other request
property), the requests that are for invariant things, then you could
arrange to /not/ proxy these requests to Tomcat, and serve them directly
from Apache httpd.


Indeed that is a good idea. We are doing that for new projects for css 
and js files (apache does not proxy certain paths and picks these up 
from the local filesystem).


We can't do that for the 100 odd legacy servers as no-one has time o 
delve into the java/JSP code. I need to do something outside of 
tomcat where possible. Just to explain, each web server is a paid-for 
project - and when it's done, it sits there for 5+ years.


Only I have the time/inclination to fix this as it's killing my VMWare 
infrastructure. Because the sites are all fronted by apache in a 
similar way, one solution is likely to apply to most of the sites.


I would also add that most of the sites are dynamically driven 
pages, even involving MySQL querying, but once launched, the data 
remains fairly static - eg GET X will always resolve to reponse Y.


I'm planning a small seminar on the value of Cache-Control for my dev 
colleagues so they can stop making this mistake ;- But that still 
leaves a lot of done projects to fix.



Which proxying method exactly are you using between Apache and Tomcat ?
(if you are using mod_proxy, then you are either using mod_proxy_http or
mod_proxy_ajp; you could also consider using mod_jk).


mod_proxy_http specifically.

mod_jk looks interesting for new projects (we have local tomcats for 
those now) - I think it may be a non-starter for old stuff as trying 
to retro fit it may not be so simple (our older tomcat servers are in 
a remote farm on their own machines hence the use of mod_proxy_http).


Shouldn't be an issue you can point the mod_jk to a remote machine - I 
do it a lot so that we can push the Tomcat application out through our 
templating output filter ... The tomcat produces a plain HTML page with 
none of the styling, and this is wrapped using our custom output filter, 
I'm guessing at this stage you can do what you want with the script...


James


Also, what are the versions of Apache and Tomcat that you are using ?



Apache 2.2 (various sub versions) and both tomcat 5.5 and tomcat 6 
(but all on remote machines listening on TCP sockets).


I think for this problem, I have to treat tomcat as a little, rather 
inefficient, black box and try to fixup on the apache front ends, 
hence the direction of my original idea...


Cheers,

Tim





--
The Wellcome Trust Sanger Institute is operated by Genome Research 
Limited, a charity registered in England with number 1021457 and a 
company registered in England with number 2742969, whose registered 
office is 215 Euston Road, London, NW1 2BE. 

RE: mod_perl output filter and mod_proxy, mod_cache

[James B. Muir]

I had to bolt on an input servlet filter to tomcat once. To do this I had to 
write the servlet filter code and then add filter and filter-mapping tags 
to the application WEB-INF/web.xml file.
-James


-Original Message-
From: Tim Watts [mailto:t...@dionic.net]
Sent: Thursday, July 14, 2011 8:12 AM
To: mod_perl list
Subject: Re: mod_perl output filter and mod_proxy, mod_cache

On 14/07/11 12:43, André Warnier wrote:
 Hi.

 I have to apologise.
 I misunderstood your first post, and I wanted to verify on the Tomcat
 list, so I quoted the following passage of your first post in my message
 there :

 Sadly, the tomcat dev's forgot to set any caching headers in the HTTP
 response (either Expires, Last-Modified or Cache-control) so the sites
 are largely uncacheable by browsers and the various tomcats are becoming
 overloaded.

 Unfortunately, the Tomcat Dev's there took it rather seriously, and as a
 consequence now you name is shit on the Tomcat list.


 .. just kidding, I did not quote your name.

LoL - I hate tomcat anyway (for it's fatness) so I don't mind if they
hate me ;-

I should have clarified as my Department's dev team (ie the ones who
use tomcat here) rather than the Tomcat Developers themselves...

I have no doubts that jsp can be told to emit certain headers but for
some reason a lot of web developers IME often miss the finer points of
HTTP. This of course would be the correct place to do it as they can
choose different max-age times to suit the content.

I plan to run a 20 minute seminar on this specific point for my lot (and
more such seminars for other issues like security and SQL efficiency)
but that still leaves loads of old black-boxes to manage for a few years.

 Anyway, apart from a few huffed responses to my misquote (since then
 rectified), someone provided a suggestion that may not be the simplest,
 but might be helpful anyway in some cases :

 Have a look at : http://www.tuckey.org/urlrewrite/

 This is a Java Servlet Filter, which can be added transparently
 around any Tomcat web application (by adding the required section in
 the web.xml config file of that web application).
 Java Servlet Filters are such that the Tomcat web application is not
 even aware that it is there, and continues to work as before. Much like
 Apache input and output filters in fact, except that a Java Servlet
 Filter is both at the same time (it wraps the webapp on both sides).

That could be interesting too - as long as it's something I can bolt in
without having to recompile the webapp code, I'm game. As a linux
sysadmin, I draw a clear line between the systems (my problem) and the
apps (dev team) - and not knowing java (much) I'm not qualified to mess
with their stuff... I'm happy to go as far as messing with server.xml
and web.xml though :)

 Anyway, this filter can do such things as conditionally or not adding
 response headers to anything the webapp produces. And it can do much
 more, as with time it has evolved into some kind of mish-mash of
 mod_rewrite, mod_headers and mod_proxy.

 It is more one-by-one work than doing something at the Apache front-end
 level or via a proxy, but it also provides better fine-tuning
 possibilities.
 So, if you can for instance easily identify the worst offenders, it
 might be an option.

 And it is certainly a good tool to have in one's toolcase.

I agree - I'll have a look at that after I play with Alex's suggestion
of Varnish :)

Thanks very much for your time :)

all the best,

Tim

--
Tim Watts
Personal Blog: http://www.dionic.net/tim/

IMPORTANT NOTICE REGARDING THIS ELECTRONIC MESSAGE:

This message is intended for the use of the person to whom it is addressed and 
may contain information that is privileged, confidential, and protected from 
disclosure under applicable law.  If you are not the intended recipient, your 
use of this message for any purpose is strictly prohibited.  If you have 
received this communication in error, please delete the message and notify the 
sender so that we may correct our records.

Re: mod_perl output filter and mod_proxy, mod_cache

[André Warnier]

Tim Watts wrote:
...



LoL - I hate tomcat anyway (for it's fatness) so I don't mind if they 
hate me ;-


I should have clarified as my Department's dev team (ie the ones who 
use tomcat here) rather than the Tomcat Developers themselves...


Well, I said that too, and said I had misquoted you, but there was little I could do about 
 that next phrase of yours :


I think for this problem, I have to treat tomcat as a little, rather inefficient, black 
box ..

Re: mod_perl output filter and mod_proxy, mod_cache

[Tim Watts]

On 14/07/11 14:38, André Warnier wrote:

Tim Watts wrote:
...



I think for this problem, I have to treat tomcat as a little, rather
inefficient, black box ..



They liked that quote then? ;-

OT Rant

I'm sure it's a lovely development environment (there must be some 
reason people use it) - all I know is it's a resource hungry bitch 
that's never happy unless it has GB's RAM and at least 2, preferably 4 
fast cores. And if you p*ss it off, it will eat your swap and burn all 
your cores at 100%. Bane of my sysadmin life...


Don't get me started on the readability of its log files!!

That's across a wide range of applications including commercial stuff 
like Confluence.


Bah - give me mod_perl (or even mod_wsgi+python) anyday...

I've got a lot done with HTML::Mason+mod_perl and very efficiently (for 
such a  simple templating system) and I've considering Mojolicious for 
fun. Learning django too right now too for the cool forms+DB stuff.


Thankfully, our guys are making a switch to django away from tomcat and 
it is so much nicer to manage.


Cheers,

Tim

--
Tim Watts
Personal Blog: http://www.dionic.net/tim/

Re [OT]: mod_perl output filter and mod_proxy, mod_cache

[André Warnier]

I'll have to watch my language here, as I might otherwise get ostracised on that other 
list of mine.


Tim Watts wrote:

On 14/07/11 14:38, André Warnier wrote:

Tim Watts wrote:
...



I think for this problem, I have to treat tomcat as a little, rather
inefficient, black box ..



They liked that quote then? ;-

OT Rant

I'm sure it's a lovely development environment (there must be some 
reason people use it) - all I know is it's a resource hungry bitch 
that's never happy unless it has GB's RAM and at least 2, preferably 4 
fast cores. And if you p*ss it off, it will eat your swap and burn all 
your cores at 100%. Bane of my sysadmin life...


We should start a club.



Don't get me started on the readability of its log files!!


Or worse, the logging configuration.



That's across a wide range of applications including commercial stuff 
like Confluence.


Bah - give me mod_perl (or even mod_wsgi+python) anyday...


+1



I've got a lot done with HTML::Mason+mod_perl and very efficiently (for 
such a  simple templating system) and I've considering Mojolicious for 
fun. Learning django too right now too for the cool forms+DB stuff.




We have been re-developing stuff that is based on , using mod_perl and TT2 
for now.
It works faster, uses umpteen MB less memory, and may soon deliver us from the management 
of that -based stuff too.



Thankfully, our guys are making a switch to django away from  and 
it is so much nicer to manage.



Don't know it, but will have a look.

[OT, ADVOCACY]

I am partial to perl and CPAN, because there are just so many things I have been able to 
do with them over the years at little expense to solve real-world problems.
And despite the fact that I also use a lot of OO modules in perl, I just cannot get in 
sympathy with a language like *, where it seems that you have to mobilise a couple of 
dozen classes (and x MB of RAM) just to print a date or so.

Never mind the time spent trying to find their documentations.

As a matter of fact, when I am confronted with a new kind of problem, in an area where I 
know a-priori nothing, my first stop is usually not Google nor Wikipedia but CPAN, just to 
read the documentation of the modules related to that area.  Whether you need to parse 
text, to process some weird data format, to talk to Amazon, to make credit-card payments, 
to dig out and generate system statistics, to understand how SOAP works, to drive an 
MS-Office program through OLE (and know nothing of OLE to start with), create a TCP 
server, convert images, read or create and send emails, or whatever, you always find an 
answer there. Even if in the end it turns out that the answer is not something in perl, 
there is so much knowledge stored in CPAN that it is a pity that it is only consulted by 
perl-centric types.


[IDEA]
Maybe creating a website named WikiPerl, containing just the CPAN documentation with a 
decent search engine (KinoSearch/Lucy ?), would help restore perl's popularity ?


Or do we just keep that for ourselves, as the best job-preservation scheme ever 
designed ?


Ooops. I was just about to send this to the wrong list...

Re: Re [OT]: mod_perl output filter and mod_proxy, mod_cache

[Niels Larsen]

Yes, CPAN has very, very useful things. I consider its biggest problems
1) too difficult to find things when not knowing what one wants, 2) a
huge undergrowth of modules that are either bad quality or unmaintained
or duplicated with a later module. The number of lingering bugs are an
obstacle, yet at the same time super-useful things are hiding in plain
view. 

Apropos, Perl Dancer was hiding for me because I didn't see it here,
http://search.cpan.org/modlist/World_Wide_Web .. but many more such 
discoveries in the past. A simple global ranking by popularity (the 
number of times downloaded) and/or by size and maturity (time located
on CPAN) would expose many new things to many, I think. If other 
modules depend on them, then that may speak to quality somewhat, and 
much better rating could be done. MongoDB would probably make managing
the collection easier. But, I am grateful for what exists of course.

While watching the language certainly, I'm moving from Apache/mod_perl
to Dancer/Nginx for speed and memory reason.

Ok, back to lurk-mode,

Niels Larsen


 [OT, ADVOCACY]
 
 I am partial to perl and CPAN, because there are just so many things I have 
 been able to 
 do with them over the years at little expense to solve real-world problems.
 And despite the fact that I also use a lot of OO modules in perl, I just 
 cannot get in 
 sympathy with a language like *, where it seems that you have to mobilise 
 a couple of 
 dozen classes (and x MB of RAM) just to print a date or so.
 Never mind the time spent trying to find their documentations.
 
 As a matter of fact, when I am confronted with a new kind of problem, in an 
 area where I 
 know a-priori nothing, my first stop is usually not Google nor Wikipedia but 
 CPAN, just to 
 read the documentation of the modules related to that area.  Whether you need 
 to parse 
 text, to process some weird data format, to talk to Amazon, to make 
 credit-card payments, 
 to dig out and generate system statistics, to understand how SOAP works, to 
 drive an 
 MS-Office program through OLE (and know nothing of OLE to start with), create 
 a TCP 
 server, convert images, read or create and send emails, or whatever, you 
 always find an 
 answer there. Even if in the end it turns out that the answer is not 
 something in perl, 
 there is so much knowledge stored in CPAN that it is a pity that it is only 
 consulted by 
 perl-centric types.
 
 [IDEA]
 Maybe creating a website named WikiPerl, containing just the CPAN 
 documentation with a 
 decent search engine (KinoSearch/Lucy ?), would help restore perl's 
 popularity ?
 
 Or do we just keep that for ourselves, as the best job-preservation scheme 
 ever designed ?
 
 
 Ooops. I was just about to send this to the wrong list...

Re: Re [OT]: mod_perl output filter and mod_proxy, mod_cache

[Clinton Gormley]

Hi Niels

On Thu, 2011-07-14 at 20:09 +0200, Niels Larsen wrote:
 Yes, CPAN has very, very useful things. I consider its biggest problems
 1) too difficult to find things when not knowing what one wants, 2) a
 huge undergrowth of modules that are either bad quality or unmaintained
 or duplicated with a later module. The number of lingering bugs are an
 obstacle, yet at the same time super-useful things are hiding in plain
 view. 

Check out http://metacpan.org - it's a GSOC 2011 project that aims to
improve cpan search.  Tagging and user ranking (plus integration of
those into the search results) are next on the feature list

clint

Re: mod_perl output filter and mod_proxy, mod_cache

[André Warnier]

Tim Watts wrote:

Hi,

Is it in theory possible to insert a perl output filter between 
mod_proxy and mod_cache?


Or at least between mod_proxy and the client?


...



mod_headers and mod_proxy don't seem to play well together and mod-cache 
doesn't either (probably due to lack of cache control headers in the 
tomcat response, though I haven't proved this is actually the case).



...

Back to the main issue.

See this as just a bit more generic information, as to what/how you could think of solving 
your problem, apart from the other suggestions already submitted.


1) I am not sure about mod_perl I/O filters, because I never used them. (*)
But in order to (conditionally/unconditionally) insert/delete/modify request/response 
headers, you can also write your own perl handler, and by choosing the appropriate type of 
 PerlHandler, you can have it run at just about any point in the request/response cycle.


The real power of mod_perl (if you haven't yet discovered that aspect), is that it allows 
you to insert your own code at just about any point of the Apache request processing 
cycle, and to do just about anything you want with any aspect of the request/response.

That includes interfering with anything that other, non-perl, Apache modules 
do.

See the following page for a good overview of the Apache request processing cycle, and 
what you can do with such PerlHandlers :

http://perl.apache.org/docs/2.0/user/handlers/intro.html#mod_perl_Handlers_Categories
You are probably more interested in the HTTP Protocol section.  By clicking on each item 
in that list, you get and explanation of /when/ that type of handle runs.

(It's also indirectly a very good introduction to how Apache itself works).

Such handlers are usually easy to write and configure, and the code to play with HTTP 
headers is also quite simple, if you know what to put in the header(s).


2) about mod_headers and mod_proxy playing together :
The trouble is that (contrarily to the mod_perl documentation above) it is not usually 
clear at all in the Apache module's documentation, to find out during which exact phase of 
the Apache request processing each module runs.


But I seem to remember something in mod_headers about an early attribute or 
parameter.
Maybe that tells you more of when it runs (or can run), compared to mod_proxy.

3) In the documentation of mod_proxy, there should be a possibility to configure it inside 
of a Location(Match) section, instead of globally (outside of any section).
That forces you to decide more finely which URLs should or should not be proxied/forwarded 
to Tomcat, but it also (in my view) makes it more evident to combine the proxying 
instruction with other modules, like perl filters or handlers.


In effect, from Apache's point of view, mod_proxy must be the equivalent of a 
content-generating handler (like a PerlResponseHandler), because for Apache, passing a 
request to mod_proxy for processing is not much different than passing it to any other 
internal response-generating handler.
Apache in fact knows nothing of Tomcat.  It passes a request to mod_proxy, and expects the 
response (or an error status) back from mod_proxy.  It has no idea that behind mod_proxy 
is another server.



4) strictly according to the HTTP protocol, a GET request should be idempotent, which 
means (roughly) that running it twice or more should always give the same answer.
Which in theory means that even if the GET request goes to a database, the response should 
be cacheable under most circumstances.
Unfortunately, the practice is such that the GET request is much overused, and it is not 
always that way.
But if caching the response creates problems, you can always tell your application 
developers that it is their fault because they are misusing the protocol..


(In really strict terms, a GET /could/ provide a different response; but it should not 
modify the state of the server).


5) despite what I am saying in (4), a GET response can very validly be different from a 
previous GET response with the same URL (for example, if in-between the data has been 
modified by a POST).  So if you are forcing headers on the responses, you should at least 
be a bit careful not to do this indiscriminately.


That is also why I personally have a doubt about the effectiveness of another caching 
proxy front-end like a couple were mentioned earlier.  If the Tomcat web applications 
themselves do not provide headers to indicate whether their response can be cached or not, 
how is the front-end going to determine that this response /is/ the same as a previous one ?
It seems to me that such a determination would require elements that such a proxy does not 
have, no ?



Now if you are still there, one more question :
Are we talking here of a configuration where one front-end Apache front-ends for several 
Tomcats possibly on different machines ?

or does each Tomcat have its own personal Apache front-end on the same machine ?
or something

Re: mod_perl output filter and mod_proxy, mod_cache

[André Warnier]

And here is another link which might be interesting.
It is a message on the Tomcat list (where I re-posted your original request, 
hem), from
Rainer Jung, who is one of the Apache/Tomcat mod_jk connector developers :


Yes, go for TC 7:

http://tomcat.apache.org/tomcat-7.0-doc/config/filter.html#Expires_Filter

Regards,

Rainer


Now that Tomcat page, apart from its own interest, also points to the Apache mod_expires 
module (which I never heard about before) in your case may be exactly what you're looking for.


It seems to be such that it can add headers in a response proxied to Tomcat, without 
overwriting such headers if they already exist.



Here is what I would do :

1) identify some usual suspects among the URLs proxied to Tomcat
   They would have to match the following criteria :
   - they happen on an overloaded Tomcat
   - they happen often
   - I am reasonably sure that the information delivered by that URL
 is stable over a period of time
   - I am reasonably sure that if it happened that the browser would,
 once in a while, get stale information, it would not be dramatic

2) carefully configure the front-end Apache to, for these particular URLs,
add an Expires header specifying now + N, where N is initially not too large.
This way, a browser would not get a result that is more than N outdated, but any duplicate 
request within a period N would get the cached version.


3) look at the impact and loop or not, increasing or decreasing N

YMMV.

Re: mod_perl output filter and mod_proxy, mod_cache

[Tim Watts]

 of the server).


I do recall that.


5) despite what I am saying in (4), a GET response can very validly be
different from a previous GET response with the same URL (for example,
if in-between the data has been modified by a POST). So if you are
forcing headers on the responses, you should at least be a bit careful
not to do this indiscriminately.

That is also why I personally have a doubt about the effectiveness of
another caching proxy front-end like a couple were mentioned earlier. If
the Tomcat web applications themselves do not provide headers to
indicate whether their response can be cached or not, how is the
front-end going to determine that this response /is/ the same as a
previous one ?
It seems to me that such a determination would require elements that
such a proxy does not have, no ?


I agree - the tomcat apps *should* be declaring what is the correct 
caching scenario. But they don't. So this is very much a work around. 
However, for any given case, the dev folk usually remember enough about 
a project to say the content of the database does not change, and GETs 
will be invariant as a result (or not). It's on that basis I'm happy to 
proceed with a kludge, just to save my poor servers from melting(!). 
Well the servers are all VMs, so in more to stop old projects stealing 
resources that could be better used on new projects.


I feel I understand Cache-Control (vs Expires) a lot better since I 
optimised my own website with mod_cache on top of HTML::Mason/mod_perl 
(which do play nice) - and my Mason bits do send sensible Cache-Control 
lines. So I plan to give a small lunchtime seminar on that topic with 
some demos of using Google's pagespeed firebug plugin (very useful for 
this stuff).


The stupid thing is, it is probably trivial at design time to wedge 
extra HTTP headers in (maybe JSP has a framework level TTL/expires 
control - I don't know) but one has to know one *should* be doing it...




Now if you are still there, one more question :
Are we talking here of a configuration where one front-end Apache
front-ends for several Tomcats possibly on different machines ?
or does each Tomcat have its own personal Apache front-end on the same
machine ?
or something in-between ?


Mix. Older projects sent 3 different VHOSTS to 3 different remote tomcat 
servers, each of which was handling a dozen+ webapps for a dozen+ 
different apache servers.


This was a disaster as one bad webapp could take out the tomcat farm and 
the bloody logs are so useless it was impossible to find out which one.


These days, we have 3 different tomcat instances on the front machine 
(dev, staging, live/production) and one apache with 3 VHOSTs mapping to 
each tomcat. We may also blend in some django on the same machine. 
Apache may mix in static content itself for efficiciency (CSS/JS).


At least then, the development tomcat can be killed and restarted 
without breaking the live one (and no, touching the web.xml file to 
trigger a single webapp reload is about reliable as asking a robber to 
drop your cash off at the bank!).


They used to use a lot of perl - but I think perl lost it a bit with 
forms handling and Ajax (until recently perhaps) which is why everyone 
went off playing with jsp and now django.


I must admit django does seem well designed and I object to python a lot 
less than java. Disadvantage - django likes to write your SQL for you 
leading to a lack of thinking there - eg, one I caught the other day:


5 JOINs with a SELECT DISTINCT over all. Bloke wondered why the MySQL 
server took 40 seconds to compute the result!




(*) considering the name of filter however, I would think that
- an input filter should always run /before/ any module which
generates content (of which mod_proxy is one)
- an output filter should always run /after/ any modules which
generate content.
So, it is probably difficult to have a filter which runs /in-between/
other Apache modules.


I'm still going to have a look at mod_perl filters - I have a feeling 
they could be useful here and there.


Thanks :)

Tim

--
Tim Watts
Personal Blog: http://www.dionic.net/tim/

Re: output filter

[William T]

On Thu, Jan 28, 2010 at 11:46 PM,  m...@normalperson.e4ward.com wrote:
 BTW, though we are still using modperl for developing some applications.
 But it seems other instead technologies are becoming more and more
 popular, like PHP,Django,Rails etc.
 Is modperl  outdated in today?

I don't know that PHP is becoming more popular, I think it's
popularity is actually waning.  While Django, and Rails are certainly
popular, I don't think it necessarily follows that modperl is
outdated.  It certainly performs better than Django and Rails, but
that isn't necessarily a hard requirement for many places.  It's
probably more of business decision these days.  Which technology do
your employees know the best, and what's the availability of people
who know that technology in the job market?  The pluses and minuses to
each language and technology mostly even out.  Chances are if you are
writing webapps your not really going to run across large
differentiating factors between the technologies.

-wjt

output filter

[moli]

Hello,

How to set the filter for some output headers?
I want to add some Expire: and Cache-Control: headers to make
downstream cacheserver to cache the pages.
Though they are dynamic pages, but I think they are safe to be cachable.

Thanks.

Re: output filter

[Torsten Förtsch]

On Thursday 28 January 2010 16:27:29 m...@normalperson.e4ward.com wrote:
 How to set the filter for some output headers?
 I want to add some Expire: and Cache-Control: headers to make
 downstream cacheserver to cache the pages.
 Though they are dynamic pages, but I think they are safe to be cachable.
 
PerlOutputFilter sub {  \
  use strict;\
  use Apache2::Filter ();\
  use Apache2::RequestRec ();\
  use Apache2::Const -compile=qw/DECLINED/; \
  my ($f)=...@_;\
  @{$f-r-headers_out}{qw/Expire Cache-Control/}=   \
(q{...}, q{...});\
  $f-remove;\
  return Apache2::Const::DECLINED;   \
}

Fill in the 2 ellipses.

Torsten

Re: output filter

[Perrin Harkins]

On Thu, Jan 28, 2010 at 10:27 AM,  m...@normalperson.e4ward.com wrote:
 I want to add some Expire: and Cache-Control: headers to make
 downstream cacheserver to cache the pages.
 Though they are dynamic pages, but I think they are safe to be cachable.

If you're generating them with mod_perl, you can just set the headers
in your response handler.  You only need an output filter if some of
them are coming from another source, like PHP.

- Perrin

Re: output filter

[moli]

Thanks all.
Yes the whole pages are generated from mod_perl not PHP.

BTW, though we are still using modperl for developing some applications.
But it seems other instead technologies are becoming more and more
popular, like PHP,Django,Rails etc.
Is modperl  outdated in today?

Regards.

Removing an output filter handler from the filters chain

[woinshet abdella]

Hello, 

I have a perl output filter handler that processes the title element of html 
documents, after I am done with processing title element, I want to remove 
the output filter handler from the output filters chain so that we do not 
process the remaining bucket brigades to improve performance. Here is the 
fragment of the script

...
if ($ctx-{done})
{ #ctx-{done} is true - we are done parsing the title, remove the handler from 
the filter chain
$f-remove;
return Apache::Const::DECLINED;
}
…
It appears that the handler is not removed from the filter chain, the incoming 
bucket brigades are still being processed by the filter.

Here is my environment 
Red Hat Enterprise Linux
Apache/2.0.46
perl, v5.8.0 
 
I would appreciate your help.

Thanks a lot.
Woinshet


  

Be a better friend, newshound, and 
know-it-all with Yahoo! Mobile.  Try it now.  
http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ

Re: Can output filter return page not found?

[Marc M. Adkins]

Thanks, good hint.  I was thinking that myself overnight.

Here's the basic scenario:

I use the main request handler to generate XML.  I have a framework that 
allows me to glue together various sub-handlers (from my framework, not 
Apache handlers) to add disparate data elements into the XML depending 
on the context of the page.


Then I use an output filter to process the XML using XSLT.  This 
completely separates data from presentation.  Works really well except 
for my current issue.  Seems like a reasonable use of the Apache 2.x 
handler structure.


So I don't actually know whether the template exists until the output 
filter.  I don't want to pollute my data generation phase with checking 
ahead for the template.  I'm also not sure why Apache isn't detecting 
that the template doesn't exist ... it should look like any other 
servable resource.


So before I start messing with headers I think I'll look at the way my 
configuration file is set up.  Perhaps I'm doing something stupid there. 
 I'm using Location instead of FilesMatch (or whatever) to 
configure the output filter ... maybe it's something like that.


Thanks again!

Adam Prime wrote:
If you're using the bucket brigade API you'd have to intercept the 
headers and modify them there.  You might have an easier time doing 
something like this using the stream API, but i don't really know, just 
taking a shot in the dark.  I'd think that it's certainly possible 
though, somehow.


That said you might be better served catching whatever situation is 
causing you to want to do this earlier in the request cycle and keeping 
the response phase from doing whatever it wants to do in the first place 
and just causing the 404 to happen then.


Adam

Marc M. Adkins wrote:
Hmmm...it's a bucket brigade FilterRequestHandler output filter.  It 
seems to not care what return code I use, whatever is in the bucket 
brigade goes out.  Can I not return Apache2::Const::NOT_FOUND from 
handler()?  That seems odd.


Marc M. Adkins wrote:
I'm trying to cause a 404 error from an output filter and it isn't 
doing what I expect.  Is this even possible?


mma

Re: Can output filter return page not found?

[deepfryed]

Sounds like something I would *not* do. If you are going to return a
404, why not do it as soon as possible.
Generating data only to return a 404 sounds like useless extra work
unless you are caching the same data for future requests.

On 12/17/07, Marc M. Adkins [EMAIL PROTECTED] wrote:
 Thanks, good hint.  I was thinking that myself overnight.

 Here's the basic scenario:

 I use the main request handler to generate XML.  I have a framework that
 allows me to glue together various sub-handlers (from my framework, not
 Apache handlers) to add disparate data elements into the XML depending
 on the context of the page.

 Then I use an output filter to process the XML using XSLT.  This
 completely separates data from presentation.  Works really well except
 for my current issue.  Seems like a reasonable use of the Apache 2.x
 handler structure.

 So I don't actually know whether the template exists until the output
 filter.  I don't want to pollute my data generation phase with checking
 ahead for the template.  I'm also not sure why Apache isn't detecting
 that the template doesn't exist ... it should look like any other
 servable resource.

 So before I start messing with headers I think I'll look at the way my
 configuration file is set up.  Perhaps I'm doing something stupid there.
   I'm using Location instead of FilesMatch (or whatever) to
 configure the output filter ... maybe it's something like that.

 Thanks again!

 Adam Prime wrote:
  If you're using the bucket brigade API you'd have to intercept the
  headers and modify them there.  You might have an easier time doing
  something like this using the stream API, but i don't really know, just
  taking a shot in the dark.  I'd think that it's certainly possible
  though, somehow.
 
  That said you might be better served catching whatever situation is
  causing you to want to do this earlier in the request cycle and keeping
  the response phase from doing whatever it wants to do in the first place
  and just causing the 404 to happen then.
 
  Adam
 
  Marc M. Adkins wrote:
  Hmmm...it's a bucket brigade FilterRequestHandler output filter.  It
  seems to not care what return code I use, whatever is in the bucket
  brigade goes out.  Can I not return Apache2::Const::NOT_FOUND from
  handler()?  That seems odd.
 
  Marc M. Adkins wrote:
  I'm trying to cause a 404 error from an output filter and it isn't
  doing what I expect.  Is this even possible?
 
  mma
 
 
 
 

Re: Can output filter return page not found?

[Adam Prime]

[EMAIL PROTECTED] wrote:

Sounds like something I would *not* do. If you are going to return a
404, why not do it as soon as possible.
Generating data only to return a 404 sounds like useless extra work
unless you are caching the same data for future requests.



I agree, if you don't want to much up your data generation code, write 
an Access or Fixup Handler (or any other pre-response phase handler of 
your choice) checks for the existence of the template at that stage.  If 
it's not there, 404.


Adam

Re: Can output filter return page not found?

[Marc M. Adkins]

I'm actually thinking the same thing.  I've been looking for a 
configuration setting that will just notice that the file isn't there 
and 404, but I think the fact that there IS a response handler is 
causing Apache to skip the check for the existence of the file.  Failing 
a simpler configuration fix I think there's no choice but to fail early 
as you both suggest.


It does seem like a different kind of wasted effort.  Now I'm going to 
check for the template in an additional handler for each and every 
request.  Since the majority of requests should be correct this is 
wasted cycles for most requests.


Failing out of the output handler wastes the data generation phase, but 
that's lightweight for most of this particular site and the percentage 
of 404 pages will likely be also.


Is:

[found-ratio] * [pre-check-time]

more or less than:

[not-found-ratio] * [data-generation-time]

?

Fortunately this is just a hobby site so I don't have to care too much. 
 And if I care, I can always instrument the code to find out the actual 
values of the three variables.


I do think that there should be a clear way to return a 404 (or any 
other code) from the output filter if desired.


mma

Adam Prime wrote:

[EMAIL PROTECTED] wrote:

Sounds like something I would *not* do. If you are going to return a
404, why not do it as soon as possible.
Generating data only to return a 404 sounds like useless extra work
unless you are caching the same data for future requests.



I agree, if you don't want to much up your data generation code, write 
an Access or Fixup Handler (or any other pre-response phase handler of 
your choice) checks for the existence of the template at that stage.  If 
it's not there, 404.


Adam

Re: Can output filter return page not found?

[Marc M. Adkins]

Implemented Fixup handler and it works fine.  Thanks all.

Marc M. Adkins wrote:
I'm actually thinking the same thing.  I've been looking for a 
configuration setting that will just notice that the file isn't there 
and 404, but I think the fact that there IS a response handler is 
causing Apache to skip the check for the existence of the file.  Failing 
a simpler configuration fix I think there's no choice but to fail early 
as you both suggest.


It does seem like a different kind of wasted effort.  Now I'm going to 
check for the template in an additional handler for each and every 
request.  Since the majority of requests should be correct this is 
wasted cycles for most requests.


Failing out of the output handler wastes the data generation phase, but 
that's lightweight for most of this particular site and the percentage 
of 404 pages will likely be also.


Is:

[found-ratio] * [pre-check-time]

more or less than:

[not-found-ratio] * [data-generation-time]

?

Fortunately this is just a hobby site so I don't have to care too much. 
 And if I care, I can always instrument the code to find out the actual 
values of the three variables.


I do think that there should be a clear way to return a 404 (or any 
other code) from the output filter if desired.


mma

Adam Prime wrote:

[EMAIL PROTECTED] wrote:

Sounds like something I would *not* do. If you are going to return a
404, why not do it as soon as possible.
Generating data only to return a 404 sounds like useless extra work
unless you are caching the same data for future requests.



I agree, if you don't want to much up your data generation code, write 
an Access or Fixup Handler (or any other pre-response phase handler of 
your choice) checks for the existence of the template at that stage.  
If it's not there, 404.


Adam

Can output filter return page not found?

[Marc M. Adkins]

I'm trying to cause a 404 error from an output filter and it isn't doing 
what I expect.  Is this even possible?


mma

Re: Can output filter return page not found?

[Marc M. Adkins]

Hmmm...it's a bucket brigade FilterRequestHandler output filter.  It 
seems to not care what return code I use, whatever is in the bucket 
brigade goes out.  Can I not return Apache2::Const::NOT_FOUND from 
handler()?  That seems odd.


Marc M. Adkins wrote:
I'm trying to cause a 404 error from an output filter and it isn't doing 
what I expect.  Is this even possible?


mma

Re: Can output filter return page not found?

[Marc M. Adkins]

Thought I had it.  Used $filter-r-status() to set the status code on 
the request from within the output filter.  That would have made sense 
and resulted in a big d'oh!  Sadly it doesn't seem to change the 
result either.


Time to do something else for a while.  Thanks in advance for any hints.

mma

Marc M. Adkins wrote:
Hmmm...it's a bucket brigade FilterRequestHandler output filter.  It 
seems to not care what return code I use, whatever is in the bucket 
brigade goes out.  Can I not return Apache2::Const::NOT_FOUND from 
handler()?  That seems odd.


Marc M. Adkins wrote:
I'm trying to cause a 404 error from an output filter and it isn't 
doing what I expect.  Is this even possible?


mma

Re: Can output filter return page not found?

[Colin Wetherbee]

Marc M. Adkins wrote:

Can I not return Apache2::Const::NOT_FOUND from handler()?


I've never used filters for this, but you can certainly return NOT_FOUND
from a handler.

Colin

Re: Can output filter return page not found?

[Adam Prime]

If you're using the bucket brigade API you'd have to intercept the 
headers and modify them there.  You might have an easier time doing 
something like this using the stream API, but i don't really know, just 
taking a shot in the dark.  I'd think that it's certainly possible 
though, somehow.


That said you might be better served catching whatever situation is 
causing you to want to do this earlier in the request cycle and keeping 
the response phase from doing whatever it wants to do in the first place 
and just causing the 404 to happen then.


Adam

Marc M. Adkins wrote:
Hmmm...it's a bucket brigade FilterRequestHandler output filter.  It 
seems to not care what return code I use, whatever is in the bucket 
brigade goes out.  Can I not return Apache2::Const::NOT_FOUND from 
handler()?  That seems odd.


Marc M. Adkins wrote:
I'm trying to cause a 404 error from an output filter and it isn't 
doing what I expect.  Is this even possible?


mma

Re: [MP2] Seg Fault in Registry script print using output filter handler

[Ed Eddington]

The workarounds (viable to me) that prevent the seg fault descirbed below
are:

1. Use LWP::UserAgent within a startup.pl (need to actually make a complete
HTTP request, not just 'use')
2. Use WWW:Curl instead of LWP::UserAgent in the output filter

One of the keys to the seg fault seems to be the use of LWP::UserAgent
within a bucket brigade filter. The seg fault occurs if Perl hasn't yet
loaded this module (and friends). After the filter runs successfully - seg
fault. Interestingly though, the other factors described are all necessary:
CGI.pm header(), the print statement must have a comma and contain 8k -
which causes the output filter to run during the print, not after.

Can anyone offer a potential cause for the seg fault? The example code
provided is all straight out of the docs.

Other things to note:
   - This occurs on a pre-forking mpm running in single process mode -X
   - The LWP request is not hitting the same server

Thanks,
Ed Eddington


On 11/15/06, [EMAIL PROTECTED] 
[EMAIL PROTECTED] wrote:



I'm getting seg faults within print of ModPerl::Registry script while
using an output filter handler. The output filter is run during the script's
print statement when the printed string is  ~8kb. Debugging shows the seg
fault occurs AFTER the output filter handler has completed successfully, but
BEFORE the print statement completes. GDB backtrace is below. Perl modules
CGI.pm and LWP::UserAgent seem to be causal in generating the seg fault.

Here is the somewhat complicated recipe that produces the seg fault
reliably.

1. ModPerl::Registry script using CGI.pm header() and a print statement
that includes a comma separated (list) of strings  8kb.

2. An output filter handler that uses LWP::UserAgent to make an HTTP
request.

Caveats:
- Only occurs in an Apache child that hasn't already loaded these
modules (CGI, LWP::UserAgent) and dependents.
- Occurs in single process mode (-X) on both static and dynamically
linked builds.

Example script/filter and system details are below. I can provide more
info if needed. I'm currently testing other module versions. Can anyone
suggest a fix or workaround?

Thanks!
*Ed Eddington*
Sr. Software Engr - Web Development
Priority Health - Grand Rapids, MI

Re: [MP2] Seg Fault in Registry script print using output filter handler

[Ed Eddington]

I found another component to the seg fault. I am using a dynamically linked
mod_deflate output filter as well (will test with static). I get no seg
fault when I comment out the DEFLATE line in the Apache config. (Sorry, I
hadn't noticed this piece of the puzzle.)

  LoadModule deflate_module  /opt/apache2/modules/mod_deflate.so
  ...
   Location /prog 
   SetHandler perl-script
   PerlResponseHandler ModPerl::Registry
   Options ExecCGI
   AddOutputFilterByType DEFLATE text/html text/plain text/xml
   /Location

[MP2] Seg Fault in Registry script print using output filter handler

[Ed . Eddington]

I'm getting seg faults within print of ModPerl::Registry script while
using an output filter handler. The output filter is run during the
script's print statement when the printed string is  ~8kb. Debugging
shows the seg fault occurs AFTER the output filter handler has completed
successfully, but BEFORE the print statement completes. GDB backtrace is
below. Perl modules CGI.pm and LWP::UserAgent seem to be causal in
generating the seg fault.

Here is the somewhat complicated recipe that produces the seg fault
reliably. 

1. ModPerl::Registry script using CGI.pm header() and a print statement
that includes a comma separated (list) of strings  8kb.
2. An output filter handler that uses LWP::UserAgent to make an HTTP
request.

Caveats:
- Only occurs in an Apache child that hasn't already loaded these
modules (CGI, LWP::UserAgent) and dependents.
- Occurs in single process mode (-X) on both static and dynamically
linked builds.

Example script/filter and system details are below. I can provide more
info if needed. I'm currently testing other module versions. Can anyone
suggest a fix or workaround? 

Thanks!
Ed Eddington
Sr. Software Engr - Web Development
Priority Health - Grand Rapids, MI


GDB BACKTRACE

(gdb) continue
Continuing.

Program received signal SIGSEGV, Segmentation fault.
Perl_do_print (my_perl=0x8497ae0, sv=0x1, fp=0x86ee6c8) at doio.c:1321
1321doio.c: No such file or directory.
in doio.c
(gdb) bt
#0  Perl_do_print (my_perl=0x8497ae0, sv=0x1, fp=0x86ee6c8) at
doio.c:1321
#1  0x00375da8 in Perl_pp_print (my_perl=0x8497ae0) at pp_hot.c:624
#2  0x00362973 in Perl_runops_debug (my_perl=0x8497ae0) at dump.c:1452
#3  0x0031d812 in S_call_body (my_perl=0x8497ae0, myop=0xbff8d050,
is_eval=0) at perl.c:2364
#4  0x0031d4dd in Perl_call_sv (my_perl=0x8497ae0, sv=0x0, flags=4) at
perl.c:2282
#5  0x0807a257 in modperl_callback (my_perl=0x8497ae0,
handler=0x92bf1d8, p=0x93522b8, r=0x93522f0, s=0x92c0ef0,
args=0x932c878)
at modperl_callback.c:100
#6  0x0807a997 in modperl_callback_run_handlers (idx=6, type=4,
r=0x93522f0, c=0x0, s=0x92c0ef0, pconf=0x0, plog=0x0, ptemp=0x0, 
run_mode=MP_HOOK_RUN_FIRST) at modperl_callback.c:261
#7  0x0807af07 in modperl_callback_per_dir (idx=6, r=0x1,
run_mode=MP_HOOK_RUN_FIRST) at modperl_callback.c:368
#8  0x08074aa1 in modperl_response_handler_run (r=0x93522f0, finish=0)
at mod_perl.c:994
#9  0x08074d2b in modperl_response_handler_cgi (r=0x93522f0) at
mod_perl.c:1089
#10 0x080f5040 in ap_run_handler ()
#11 0x080f579e in ap_invoke_handler ()
#12 0x080c55da in ap_process_request ()
#13 0x080bfa06 in ap_process_http_connection ()
#14 0x08101ab8 in ap_run_process_connection ()
#15 0x08101ea3 in ap_process_connection ()
#16 0x080f38fb in child_main ()
#17 0x080f39d2 in make_child ()
#18 0x080f3af8 in startup_children ()
#19 0x080f3efe in ap_mpm_run ()
#20 0x080fac06 in main ()



EXAMPLE SCRIPT

#!/usr/bin/perl
use strict;
use Carp;

use CGI qw(:standard);

warn TEST START...;

# doesn't matter if you have this line in the print stmt or not
header(-type=text/html);

my $head = 'HTML';
my $foot = '/html';
my $string = string;

print $head,$string,br,$foot;

warn TEST DONE--;

sub string {
my $string;
for (1..8200) {
$string.= 'A';
}
return $string;
}


EXAMPLE FILTER:

#file:Apache2/SimpleFilter.pm
#
package PH::Apache2::SimpleFilter;

use strict;
use warnings;

use base qw(Apache2::Filter);

use APR::Brigade ();
use APR::Bucket ();

use Apache2::Const -compile = 'OK';
use APR::Const -compile = ':common';

sub handler : FilterRequestHandler {
my ($f, $bb) = @_;

my $bb_ctx = APR::Brigade-new($f-c-pool, $f-c-bucket_alloc);

while (!$bb-is_empty) {
my $b = $bb-first;

$b-remove;

if ($b-is_eos) {
$bb_ctx-insert_tail($b);
last;
}

if ($b-read(my $data)) {
#Do Nothing
#$data = join ,
#map {scalar(reverse $_), \n} split \n, $data;
$b = APR::Bucket-new($bb-bucket_alloc, $data);
}
use LWP::UserAgent;
my $ua = LWP::UserAgent-new();
my $uri = 'http://www.yahoo.com';
my $response = $ua-get($uri);
warn LWP got $response;

$bb_ctx-insert_tail($b);
  }

  my $rv = $f-next-pass_brigade($bb_ctx);
  return $rv unless $rv == APR::Const::SUCCESS;

  Apache2::Const::OK;
}
1;

-
SYSTEM DETAILS:
-

mod_perl version 2.01
LWP - 5.803
LWP::UserAgent - 2.03
CGI - 3.25
Apache2::Request - 2.08


$ uname -a
Linux molly 2.6.9-42.ELsmp #1 SMP Wed Jul 12 23:27:17 EDT 2006 i686 i686
i386 GNU/Linux


DYNAMIC APACHE

$ /opt/apache2/bin/httpd -l
Compiled in modules:
  core.c
  prefork.c
  http_core.c
  mod_so.c

$ /opt/apache2/bin/httpd -V
Server version: Apache/2.0.59
Server built:   Jul 28 2006 15:42:08
Server's Module Magic Number

[mp2] bb output filter that changes content length

[Beau E. Cox]

Hi all -

I am trying to change the content-length in an output filter, following the
example in the mp2 docs:

http://perl.apache.org/docs/2.0/user/handlers/filters.html#Setting_the_Content_Length_Header_in_Request_Output_Filters

I have copied the example _exactly_, except:

 - use $f-ctx inplace rather than a separate $ctx.
 - omitted the '$bdata =~ s/-//g;' line (what the heck is that?)
 - added some STDERR debugging prints:
for( my $b = $bb-first; $b; $b = $bb-next( $b ) ) {
print STDERR invoked: , $f-ctx-{invoked},  bucket type: 
,b-type-name;
$seen_eos++, last if $b-is_eos;
$b-read( my $bdata );
print STDERR  data: [$bdata]\n;
push @data, $bdata;
}

Works fine for normal pages; however in an error (NOT FOUND, etc)
I get lots of duplicate bucket data. Here is a snippet of a 404:

[... snipped a bunch of empty buckets ...]
invoked: 11 bucket type: MMAP data: []
invoked: 11 bucket type: MMAP data: []
invoked: 11 bucket type: MMAP data: [?xml version=1.0 encoding=]
invoked: 12 bucket type: MMAP data: []
invoked: 12 bucket type: MMAP data: []
invoked: 12 bucket type: MMAP data: []
invoked: 12 bucket type: MMAP data: []
invoked: 12 bucket type: MMAP data: []
invoked: 12 bucket type: MMAP data: []
invoked: 12 bucket type: MMAP data: [?xml version=1.0 encoding=]
invoked: 12 bucket type: POOL data: [ISO-8859-1]
invoked: 12 bucket type: MMAP data: [?
!DOCTYPE html PUBLIC -//W3C//DTD XHTML 1.0 Strict//EN
  http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd;
html xmlns=http://www.w3.org/1999/xhtml; lang=]
invoked: 13 bucket type: MMAP data: []
invoked: 13 bucket type: MMAP data: []
invoked: 13 bucket type: MMAP data: []
invoked: 13 bucket type: MMAP data: []
invoked: 13 bucket type: MMAP data: []
invoked: 13 bucket type: MMAP data: []
invoked: 13 bucket type: MMAP data: [?xml version=1.0 encoding=]
invoked: 13 bucket type: POOL data: [ISO-8859-1]
invoked: 13 bucket type: MMAP data: [?
!DOCTYPE html PUBLIC -//W3C//DTD XHTML 1.0 Strict//EN
  http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd;
html xmlns=http://www.w3.org/1999/xhtml; lang=]
invoked: 13 bucket type: POOL data: [en]
invoked: 13 bucket type: MMAP data: [ xml:lang=]
[... snipped the rest ...]

Needless to say the returned page is trashed. What am I missing?
At the bottom of your filters tutorial, you mention some important
consideratons about checking the bucket type before saving it, but
I can't really see how this works.

Oh, I am using mp2 svn, apache 2.3 trunk svn, perl 5.8.7, on a
Gentoo Linux box.

Aloha = Beau;

Re: setting http output filter priority?

[Jeff Ambrosino]

I'm logging some closure to this one...  I was able to get this to
work by modifying the priorities of
mod_perl/src/modules/perlmod_perl.c's http output filter to
AP_FTYPE_CONTENT_SET-1, and then
httpd-2.0.53/modules/experimental/mod_cache.c's cache_save and
cache_out filter to AP_FTYPE_CONTENT_SET-2.  Everything works! :)

Note this change impacts all mod_perl apps running on your system. 
There are no other conflicts with Apache's built-in handlers and
filters that I could see, although that could change in the future.  I
also think it might be a useful feature to be able to set mod_perl's
filter priority either programmatically, or at least as a compile-time
option.

thanks
Jeff


On Apr 2, 2005 9:09 AM, Geoffrey Young [EMAIL PROTECTED] wrote:
 
  So, mod_perl.c could register as AP_FTYPE_CONTENT, and mod_deflate.c could
  register as AP_FTYPE_CONTENT_SET+1. I think this is the least invasive
  change to get
  things working...  do you concur?
 
 off the top of my head that looks ok.  I just hope it doesn't muck with any
 additional filters you may need later on.
 
 good luck :)
 
 --Geoff

Re: setting http output filter priority?

[Geoffrey Young]

Jeff Ambrosino wrote:
 I
 also think it might be a useful feature to be able to set mod_perl's
 filter priority either programmatically, or at least as a compile-time
 option.

yes, we've discussed both of these options on the dev list, so you can check
the archives for all the issues (look for something about dynamic hook
ordering on dev@ and [EMAIL PROTECTED]).  basically, for it to work
dynamically across all platforms we need some changes in httpd, so you
wouldn't get that in 2.0 for sure.  and I'm not sure that compile-time
options are the best recourse.

but I feel your pain - the entire hooking mechanism leaves knowing users
without a shuffle option, and that isn't very nice.

--Geoff

Re: setting http output filter priority?

[Geoffrey Young]

 So, mod_perl.c could register as AP_FTYPE_CONTENT, and mod_deflate.c could
 register as AP_FTYPE_CONTENT_SET+1. I think this is the least invasive
 change to get
 things working...  do you concur?

off the top of my head that looks ok.  I just hope it doesn't muck with any
additional filters you may need later on.

good luck :)

--Geoff

Re: setting http output filter priority?

[Geoffrey Young]

 See the problem?  MYFILTER is getting inserted before the cache is serving up
 the content, but I really want it to go after (otherwise MYFILTER doesn't
 see the body content).  Is there a way around this aside from
 modifying mod_cache?

at the moment, no - the hooking mechanism in apache just isn't designed for
this type of thing.  in apache 1.3 we could mess with the order in which
modules ran, but not in 2.0.

an alternative to modifying mod_cache would be to modify mod_perl.  from the
look of things, mod_cache needs to be in a specific place wrt other core
modules, so it might be easier to just modify mod_perl.  but that would
affect the order of all perl filters, so...

--Geoff

Re: setting http output filter priority?

[Jeff Ambrosino]

From looking at the code, it appears that I may need to modify
mod_perl and either
of mod_cache or mod_deflate:

mod_perl.c - line ~839
--
ap_register_output_filter(MP_FILTER_REQUEST_OUTPUT_NAME,
  MP_FILTER_HANDLER(modperl_output_filter_handler),
  AP_FTYPE_RESOURCE);   // 
---

mod_cache.c - line ~936
--
cache_save_filter_handle =
ap_register_output_filter(CACHE_SAVE,
  cache_save_filter,
  NULL,
  AP_FTYPE_CONTENT_SET-1);  // 
---
[...]
cache_out_filter_handle =
ap_register_output_filter(CACHE_OUT,
  cache_out_filter,
  NULL,
  AP_FTYPE_CONTENT_SET-1);  // 
---

mod_deflate.c - line ~845
--
ap_register_output_filter(deflateFilterName, deflate_out_filter, NULL,
  AP_FTYPE_CONTENT_SET);// 
---

So, mod_perl.c could register as AP_FTYPE_CONTENT, and mod_deflate.c could
register as AP_FTYPE_CONTENT_SET+1. I think this is the least invasive
change to get
things working...  do you concur?

thanks,
Jeff


On Apr 1, 2005 7:02 PM, Geoffrey Young [EMAIL PROTECTED] wrote:
 
  See the problem?  MYFILTER is getting inserted before the cache is serving 
  up
  the content, but I really want it to go after (otherwise MYFILTER doesn't
  see the body content).  Is there a way around this aside from
  modifying mod_cache?
 
 at the moment, no - the hooking mechanism in apache just isn't designed for
 this type of thing.  in apache 1.3 we could mess with the order in which
 modules ran, but not in 2.0.
 
 an alternative to modifying mod_cache would be to modify mod_perl.  from the
 look of things, mod_cache needs to be in a specific place wrt other core
 modules, so it might be easier to just modify mod_perl.  but that would
 affect the order of all perl filters, so...
 
 --Geoff

setting http output filter priority?

[Jeff Ambrosino]

Hi mod_perl folks,

background: I have an http output filter working on Apache 2.0.53 with
mod_proxy, mod_cache, deflate and mod_perl 2.0.0-RC4.  My functionality
is that I'm stripping out certain types of advertising content, IMG
tags, etc.

The problem I'm seeing is that if a request can be fulfilled with content
from mod_cache, then the content itself (body) isn't available to my 
http output filter...  it's only available if the content is NOT found 
in the cache.  The browser is getting the response fine, so I know the
proxy and cache are working, it's just that my filter not being inserted
in the right place.  So I'm trying to figure out how to insert my filter
after mod_cache has found/stored the content, but before defalte.

I've studied the MP2 docs (including the 3 books I have: mp Cookbook,
Lincoln Stein's and Stas Bekman's), but really can't find anything that
tells me precisely how to insert an MP2 filter with a specific AP_FTYPE_*.

Here's a simple flow of what I'd like to have happen:

1 - mod_cache [got valid content in cache? If so, go to 4; if not, go to 2]
2 - mod_proxy [fetch content from origin web]
3 - mod_cache [content cacheable?  If so, cache it locally]
4 - *MY FILTER*
5 - deflate

How can I get MYFILTER inserted in just the right place?  (And ideally it
doesn't require modifying Apache source code to change AP_FTYPE values
for mod_cache...!)

thanks
Jeff

Re: setting http output filter priority?

[Geoffrey Young]

 I've studied the MP2 docs (including the 3 books I have: mp Cookbook,
 Lincoln Stein's and Stas Bekman's), but really can't find anything that
 tells me precisely how to insert an MP2 filter with a specific AP_FTYPE_*.
 
 Here's a simple flow of what I'd like to have happen:
 
 1 - mod_cache [got valid content in cache? If so, go to 4; if not, go to 2]
 2 - mod_proxy [fetch content from origin web]
 3 - mod_cache [content cacheable?  If so, cache it locally]
 4 - *MY FILTER*
 5 - deflate
 
 How can I get MYFILTER inserted in just the right place?  (And ideally it
 doesn't require modifying Apache source code to change AP_FTYPE values
 for mod_cache...!)

try PerlSetOutputFilter

http://perl.apache.org/docs/2.0/user/handlers/filters.html#C_PerlSetOutputFilter_

HTH

--Geoff

PerlMagick in an output filter

[Dean Maslic]

Hi,

Im trying to implement an output filter that modifies images using
PerlMagick in a proxy environment.

Since magick reads an image from a file-handle/stdin I guess I need to
write the response data to a temp-file, Im not sure if its better to
use the stream api's $f-read(my $buffer, BUFF_LEN) or the
bucket_brigades to loop thru the data and print to temp_file, or it
doesn't matter ?

Once Im done with modifying the image object, how do I return the data
to the browser? Can I just write to stdout (.$image-Write(png:-);)
and somehow signal to Apache to skip any other filters in the
pipeline?

Help appreciated

Dean

Re: PerlMagick in an output filter

[Stas Bekman]

Dean Maslic wrote:
Hi,
Im trying to implement an output filter that modifies images using
PerlMagick in a proxy environment.
Since magick reads an image from a file-handle/stdin I guess I need to
write the response data to a temp-file, Im not sure if its better to
use the stream api's $f-read(my $buffer, BUFF_LEN) or the
bucket_brigades to loop thru the data and print to temp_file, or it
doesn't matter ?
Dean, conceptionally it doesn't matter. The filter read() API internally 
implements bucket brigades traversal. It's just with read() API you aren't 
as flexible as with direct bb manipulation.

Ideally there should be a PerlIO layer/or TIE interface that does all the 
work for you, so $filter can be also used as a filehandle, from which you 
can read and write normally. I believe it's somewhere on our long term 
todo list, but you are always more than welcome send patches to implement 
it sooner.

Once Im done with modifying the image object, how do I return the data
to the browser? Can I just write to stdout (.$image-Write(png:-);)
and somehow signal to Apache to skip any other filters in the
pipeline?
If you decided to use the filter, you send the data out the same way you 
got it in, depending on which of the two filter kinds you've chosen 
(streamed or bb). See the filter docs for more info:
http://perl.apache.org/docs/2.0/user/handlers/filters.html
And please ask more specific questions if still in doubt.

--
__
Stas BekmanJAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide --- http://perl.apache.org
mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org   http://ticketmaster.com

Re: Bug: mod_perl and mod_xslt as output filter

[Stas Bekman]

Stas Bekman wrote:
Gregory Burmistrov wrote:
Hello!
I've had a challenge with mod_perl and mod_xslt under Apache2, the
problem is:
mod_xslt collects buckets from mod_perl for further processing. When
mod_xslt got EOS bucket processing occurs, but some data in buckets
appear to be corrupted.
After few hours with various server configurations, gdb and other fun
stuff, I've found a clue. At least it works for me.

I see what you mean Gregory. But that solution doesn't make me happy. 
Most of the time you won't need to do that, and forcing a double copy of 
data on each print call is not an efficient thing to do. Let me ask 
httpd-dev whether there is a way to mark buckets as something that has 
to be copied on if set-aside.
This time the answer came back quicky, quoting Bill:
  It's a bug in mod_xslt, if that module trys to set aside a transient bucket.
http://marc.theaimsgroup.com/?t=10849435502r=1w=2
So please tell the author of mod_xslt that it needs to check whether the 
bucket type is transient and if so it has to do the copying on itself. 
Unfortunately I can't point you to the httpd filter docs, since I'm not aware 
of their existence. Though apr_bucket.h has this:

/**
 * The TRANSIENT bucket type.  This bucket represents a data allocated off
 * the stack.  When the setaside function is called, this data is copied on
 * to the heap
 */
APU_DECLARE_DATA extern const apr_bucket_type_t apr_bucket_type_transient;
So if mod_xslt uses a proper function like: apr_bucket_setaside to keep the 
buckets, supposedly it should handle the copying on itself, transparently to 
the filter writer.

I'll document that in our source code and have a note for modperl filter writers.
--
__
Stas BekmanJAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide --- http://perl.apache.org
mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org   http://ticketmaster.com
--
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html

RE: Bug: mod_perl and mod_xslt as output filter

[Gregory Burmistrov]

Hello!

You are right! Double copy is a bad idea and mod_xslt should be fixed.
I can't provide path for mod_xslt now because it's already hacked for my
needs. May be later. The author of mod_xslt didn't answer at all.

Best regards,
/grig

-Original Message-
From: Stas Bekman [mailto:[EMAIL PROTECTED] 
Sent: Wednesday, May 19, 2004 10:28 AM
Cc: Gregory Burmistrov; [EMAIL PROTECTED]
Subject: Re: Bug: mod_perl and mod_xslt as output filter

Stas Bekman wrote:
 Gregory Burmistrov wrote:
 
 Hello!

 I've had a challenge with mod_perl and mod_xslt under Apache2, the
 problem is:

 mod_xslt collects buckets from mod_perl for further processing. When
 mod_xslt got EOS bucket processing occurs, but some data in buckets
 appear to be corrupted.

 After few hours with various server configurations, gdb and other fun
 stuff, I've found a clue. At least it works for me.
 
 
 I see what you mean Gregory. But that solution doesn't make me happy. 
 Most of the time you won't need to do that, and forcing a double copy
of 
 data on each print call is not an efficient thing to do. Let me ask 
 httpd-dev whether there is a way to mark buckets as something that has

 to be copied on if set-aside.

This time the answer came back quicky, quoting Bill:

   It's a bug in mod_xslt, if that module trys to set aside a transient
bucket.

http://marc.theaimsgroup.com/?t=10849435502r=1w=2

So please tell the author of mod_xslt that it needs to check whether the

bucket type is transient and if so it has to do the copying on itself. 
Unfortunately I can't point you to the httpd filter docs, since I'm not
aware 
of their existence. Though apr_bucket.h has this:

/**
  * The TRANSIENT bucket type.  This bucket represents a data allocated
off
  * the stack.  When the setaside function is called, this data is
copied on
  * to the heap
  */
APU_DECLARE_DATA extern const apr_bucket_type_t
apr_bucket_type_transient;

So if mod_xslt uses a proper function like: apr_bucket_setaside to keep
the 
buckets, supposedly it should handle the copying on itself,
transparently to 
the filter writer.

I'll document that in our source code and have a note for modperl filter
writers.

-- 
__
Stas BekmanJAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide --- http://perl.apache.org
mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org   http://ticketmaster.com

--
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html

Bug: mod_perl and mod_xslt as output filter

[Gregory Burmistrov]

Hello!

I've had a challenge with mod_perl and mod_xslt under Apache2, the
problem is:

mod_xslt collects buckets from mod_perl for further processing. When
mod_xslt got EOS bucket processing occurs, but some data in buckets
appear to be corrupted.

After few hours with various server configurations, gdb and other fun
stuff, I've found a clue. At least it works for me.

--- patch ---
*** modperl_filter.c.orig   Tue May 18 17:37:34 2004
--- modperl_filter.cTue May 18 17:38:06 2004
*** MP_INLINE apr_status_t modperl_wbucket_p
*** 127,132 
--- 127,133 
  apr_bucket_alloc_t *ba = (*wb-filters)-c-bucket_alloc;
  apr_bucket_brigade *bb;
  apr_bucket *bucket;
+ char *copy;

  /* reset the counter to 0 as early as possible and in one place,
   * since this function will always either pass the data out (and
*** MP_INLINE apr_status_t modperl_wbucket_p
*** 165,171 
  }

  bb = apr_brigade_create(wb-pool, ba);
! bucket = apr_bucket_transient_create(buf, len, ba);
  APR_BRIGADE_INSERT_TAIL(bb, bucket);

  if (add_flush_bucket) {
--- 166,173 
  }

  bb = apr_brigade_create(wb-pool, ba);
! copy = apr_pmemdup(wb-pool, buf, len);
! bucket = apr_bucket_transient_create(copy, len, ba);
  APR_BRIGADE_INSERT_TAIL(bb, bucket);

  if (add_flush_bucket) {



Best regards,
/grig

P.S. mod_xslt-2.0.4 mod_perl-1.99.13 apache-2.0.49

--
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html

Re: Bug: mod_perl and mod_xslt as output filter

[Stas Bekman]

Gregory Burmistrov wrote:
Hello!
I've had a challenge with mod_perl and mod_xslt under Apache2, the
problem is:
mod_xslt collects buckets from mod_perl for further processing. When
mod_xslt got EOS bucket processing occurs, but some data in buckets
appear to be corrupted.
After few hours with various server configurations, gdb and other fun
stuff, I've found a clue. At least it works for me.
I see what you mean Gregory. But that solution doesn't make me happy. Most of 
the time you won't need to do that, and forcing a double copy of data on each 
print call is not an efficient thing to do. Let me ask httpd-dev whether there 
is a way to mark buckets as something that has to be copied on if set-aside.

! copy = apr_pmemdup(wb-pool, buf, len);
! bucket = apr_bucket_transient_create(copy, len, ba);
Also in the future please post unified diffs 'cvs diff -u'. Thanks.

__
Stas BekmanJAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide --- http://perl.apache.org
mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org   http://ticketmaster.com
--
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html

Re: Calling perl output filter before Includes filter

[david]

you could always just do a global search and replace on all the files
instead of defering this step to serving time
--
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html

Re: Calling perl output filter before Includes filter

[Geoffrey Young]

Hamilton, Henrique H wrote:
 I am using apache2.0.48 and mod perl 1.99_12.
 I've written a mod perl module as a perl output filter that rewrites image
 links
 to point to another server.[example: img src=header.gif becomes img
 src=//someserver.net/www.mysite.com/header.gif]
 My problem is that my perl module doesn't play well with SSI's . 
 I know if I call the perl module before it's processed by the ssi (INCLUDES
 filter) everything will work fine.

that sounds like a problem with your logic, not a problem with your filter
order :)

 Is there a way to force my perl output filter handler to be called BEFORE
 the html is parsed by the INCLUDES output filter?

see PerlSetOuputFilter

http://perl.apache.org/docs/2.0/user/handlers/filters.html#C_PerlSetOutputFilter_

HTH

--Geoff





-- 
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html