Re: [mp2] CGI redirects incorrectly handled?
Stas Bekman wrote:
So can flushing be held off until either (1) blank line is printed,
(2) the 8k buffer fills, or (3) send_http_header is called?
1) is relevant only for handler that print headers, rather than set them
2) absolutely not, what if you want to flush data before?
3) send_http_header doesn't exist in Apache2/mod_perl2
I didn't realise that mp2 doesn't use send_http_header. That explains
the appearance of wb-r-content_type in the mp2 code. So is it
true that if headers are sent using the API then no output filtering
and transmission occurs until the 8k buffer is either filled or flushed
(explcitly or after exit)?
Only in the case that your handler is configured with:
PerlOptions +ParseHeaders
*and*
it prints headers ala:
print Content-type:
In all other cases where headers are set via the API, e.g.
$r-content_type, $r-headers_out, etc, there is no such a thing as the
handler has send an empty line signaling the end of sending headers,
because it never sends any headers at all, but uses api to set them.
Is +ParseHeaders always indicative of explicit header printing, or
can it also be set when using the API? If the former, then yes, if
+ParseHeaders is set flushing could be held off until a blank line is seen.
With the current mp2 code, if you decide to
wait for the end of headers before doing cgi parsing and flushing then
the code is assuming that either the headers are less than 8k or that any
Status header is in the first 8k. Otherwise the code would have to
be re-written to use continuous (spilling and merging) buffer buckets
like mod_cgi. It can hold off on flushing indefinitely.
That approach will break this handler:
sub handler {
my $r = shift;
$r-content_type('text/plain');
$r-rflush; # send something to the client immediately
long_job();
return Apache::OK
}
However notice that it doesn't have to set content_type() because some
earlier handler could have done that and that should work as well:
sub handler {
my $r = shift;
$r-rflush; # send something to the client immediately
long_job();
return Apache::OK
}
So as you can see, this handler doesn't tell us when it's done with
headers.
OK, you may say that that previous handler should have marked the end of
headers settings, but that would be wrong if the response handler wants
to set other headers as well.
Do we now agree that the event of end of sending headers is possible
only in the case explained at the top?
Yes, the key I was missing is that mp2 no longer uses send_http_header.
Can you just lock out flushing when nothing has been printed and
content_type is undefined? (You impliy above that the content_type
setting is persistent, so the script would have to undef it if necessary.)
Then all the user script has to do is to make sure any Status header is
either printed or set via headers_out in the first batch of
printing/setting code before flush is called (again).
Re: [mp2] CGI redirects incorrectly handled?
Mark James wrote: Stas Bekman wrote: So can flushing be held off until either (1) blank line is printed, (2) the 8k buffer fills, or (3) send_http_header is called? 1) is relevant only for handler that print headers, rather than set them 2) absolutely not, what if you want to flush data before? 3) send_http_header doesn't exist in Apache2/mod_perl2 I didn't realise that mp2 doesn't use send_http_header. That explains the appearance of wb-r-content_type in the mp2 code. I'm not quite happy yet about the current situation with send_http_header API removal. Currently an explicit call to $r-content_type (only in mod_perl handlers) turns the headers parsing off, if it was on. Which works fine as a replacement for $r-send_http_header. However it's possible that some earlier phase calls $r-content_type and the response phase still wants to *print* its headers, rather than use API to set them. If that's the case, we are in trouble. I'll be soon working on providing the API for PerlOptions and other config options. And while most of these things are read-only, I'm thinking that I might be able to add a read/write accessor for ParseHeaders. So one can turn the parsing on and off, disregarding what was the setting in httpd.conf. I believe that would be the perfect solution, since it'll give developers a total flexibility. So is it true that if headers are sent using the API then no output filtering and transmission occurs until the 8k buffer is either filled or flushed (explcitly or after exit)? That's correct. Though the headers are really sent by the Apache core http output filter, once we send the response body. Only in the case that your handler is configured with: PerlOptions +ParseHeaders *and* it prints headers ala: print Content-type: In all other cases where headers are set via the API, e.g. $r-content_type, $r-headers_out, etc, there is no such a thing as the handler has send an empty line signaling the end of sending headers, because it never sends any headers at all, but uses api to set them. Is +ParseHeaders always indicative of explicit header printing, or can it also be set when using the API? If the former, then yes, if +ParseHeaders is set flushing could be held off until a blank line is seen. See my plan and current situation explanation above. Do we now agree that the event of end of sending headers is possible only in the case explained at the top? Yes, the key I was missing is that mp2 no longer uses send_http_header. Can you just lock out flushing when nothing has been printed and content_type is undefined? (You impliy above that the content_type setting is persistent, so the script would have to undef it if necessary.) Then all the user script has to do is to make sure any Status header is either printed or set via headers_out in the first batch of printing/setting code before flush is called (again). As I suggested earlier, I think the solution is to ignore rflush calls if we expect to parse headers and they weren't parsed yet. However if the buffer overflows, we have not much choice but to send it out. But I think that this will be a satisfactory solution. Also since close($fh) always flushes in mod_cgi, I think you can get rid of the explicit flush for good. Since mod_perl's CLOSE is no-op, it won't cause the flush (at least for now). __ Stas BekmanJAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide --- http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com
Re: [mp2] CGI redirects incorrectly handled?
Stas Bekman wrote: Mark James wrote: STDOUT is flushed prior to a fork to exec an external binary (rcs). I understand the cause. But I hope that you agree with me that this is an application's problem. If you haven't sent anything to STDOUT yet, don't flush. And if this is not under your control, reopen STDOUT to /dev/null before you call that piece of code, that flushes and then re-tie STDOUT again. (See t/response/TestModperl/request_rec_tie_api.pm) I guess the best way to fix the problem in-application would be to either run nph, or do the /dev/null redirect you suggest. Interestingly, commenting out the pre-fork flush fixes the problem under mod_perl because close in mod_perl seems to be a no-op rather than a flush. If the close is also no problem under mod_cgi then is there any real need for such a pre-fork flush in my script? I see. But why is there no problem when using mod_cgi? That's an interesting question. mod_cgi is a generic handler, which can run applications written in any language. Therefore it has no clue of what flush is. It simply creates a pipe to the application, and expects the headers headers followed by the data. In your case, when cgi script flushes STDOUT, nothing happens at all, because there is no data to flush. So mod_cgi gets the headers and the data and all is cool When the same code is run under mod_perl, flush generates a special bucket which is sent out to the filters chain, and since no headers are generated yet, they get generated and sent out. Well, even under mod_cgi a program can still fflush or write. The difference between mod_cgi and mod_perl is that mod_cgi does not activate the filter brigade until it has read all the headers. Why would a perl handler script want to flush data down the filter chain before it had finished writing all of it? Here is an example: You have a long running process, you want the headers to be sent immediately, but the data won't follow for a while. So you create the headers, do $r-rflush, and later on send the data. OK. So would there be a problem if mod_perl waited for the blank line end of headers, or at least a Status header, before passing the buckets down the line, just like mod_cgi? mod_cgi uses spilling buckets, each of size 8K, to buffer script output, including during the header scan, while mod_perl seems to scan the headers when the first 8K buffer is either filled or flushed.
Re: [mp2] CGI redirects incorrectly handled?
Mark James wrote: Stas Bekman wrote: Mark James wrote: STDOUT is flushed prior to a fork to exec an external binary (rcs). I understand the cause. But I hope that you agree with me that this is an application's problem. If you haven't sent anything to STDOUT yet, don't flush. And if this is not under your control, reopen STDOUT to /dev/null before you call that piece of code, that flushes and then re-tie STDOUT again. (See t/response/TestModperl/request_rec_tie_api.pm) I guess the best way to fix the problem in-application would be to either run nph, or do the /dev/null redirect you suggest. Interestingly, commenting out the pre-fork flush fixes the problem under mod_perl because close in mod_perl seems to be a no-op rather than a flush. If the close is also no problem under mod_cgi then is there any real need for such a pre-fork flush in my script? Since the mod_perl's internal STDOUT buffer isn't mangled if you re-tie it later, and it'll be always flushed at the end of the request, there is no *need* to flush on CLOSE. However in order to be consistent with perl fh close behavior, it probably needs to be changed to flush its buffer. What do you think? I see. But why is there no problem when using mod_cgi? That's an interesting question. mod_cgi is a generic handler, which can run applications written in any language. Therefore it has no clue of what flush is. It simply creates a pipe to the application, and expects the headers headers followed by the data. In your case, when cgi script flushes STDOUT, nothing happens at all, because there is no data to flush. So mod_cgi gets the headers and the data and all is cool When the same code is run under mod_perl, flush generates a special bucket which is sent out to the filters chain, and since no headers are generated yet, they get generated and sent out. Well, even under mod_cgi a program can still fflush or write. Ah, of course! The difference between mod_cgi and mod_perl is that mod_cgi does not activate the filter brigade until it has read all the headers. But in the case of mod_perl, this event is valid only for handlers which print their own headers, rather than using mod_perl API to set them. In the generic case, there is no way to tell whether a handler is going to set more headers or it has done with it. I suppose that we could prevent flushing in the case the handler is configured to parse headers. Does it make sense? Why would a perl handler script want to flush data down the filter chain before it had finished writing all of it? Here is an example: You have a long running process, you want the headers to be sent immediately, but the data won't follow for a while. So you create the headers, do $r-rflush, and later on send the data. OK. So would there be a problem if mod_perl waited for the blank line end of headers, or at least a Status header, before passing the buckets down the line, just like mod_cgi? See above. mod_cgi uses spilling buckets, each of size 8K, to buffer script output, including during the header scan, while mod_perl seems to scan the headers when the first 8K buffer is either filled or flushed. I don't think this is related to the issue in question. Since the problem is what to do on flush. Also we might have a problem if the headers to parse are bigger than the size of the buffer (8k). Do you think someone will ever need to send headers bigger than 8k? __ Stas BekmanJAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide --- http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com
Re: [mp2] CGI redirects incorrectly handled?
Stas Bekman wrote: Since the mod_perl's internal STDOUT buffer isn't mangled if you re-tie it later, and it'll be always flushed at the end of the request, there is no *need* to flush on CLOSE. However in order to be consistent with perl fh close behavior, it probably needs to be changed to flush its buffer. What do you think? Dunno. But the problem I had would have been even harder to track down if commenting out the flush hadn't fixed it. The difference between mod_cgi and mod_perl is that mod_cgi does not activate the filter brigade until it has read all the headers. But in the case of mod_perl, this event is valid only for handlers which print their own headers, rather than using mod_perl API to set them. In the generic case, there is no way to tell whether a handler is going to set more headers or it has done with it. So can flushing be held off until either (1) blank line is printed, (2) the 8k buffer fills, or (3) send_http_header is called? I suppose that we could prevent flushing in the case the handler is configured to parse headers. Does it make sense? No. Could you explain your reasoning. mod_cgi uses spilling buckets, each of size 8K, to buffer script output, including during the header scan, while mod_perl seems to scan the headers when the first 8K buffer is either filled or flushed. I don't think this is related to the issue in question. Since the problem is what to do on flush. Also we might have a problem if the headers to parse are bigger than the size of the buffer (8k). Do you think someone will ever need to send headers bigger than 8k? Yes, I mentioned the buffer size in case your objection to my proposal to wait until end of headers was seen was based on the possiblity of more than 8k of headers. With the current mp2 code, if you decide to wait for the end of headers before doing cgi parsing and flushing then the code is assuming that either the headers are less than 8k or that any Status header is in the first 8k. Otherwise the code would have to be re-written to use continuous (spilling and merging) buffer buckets like mod_cgi. It can hold off on flushing indefinitely.
Re: [mp2] CGI redirects incorrectly handled?
The
difference between mod_cgi and mod_perl is that mod_cgi does not
activate the filter brigade until it has read all the headers.
But in the case of mod_perl, this event is valid only for handlers
which print their own headers, rather than using mod_perl API to set
them. In the generic case, there is no way to tell whether a handler
is going to set more headers or it has done with it.
So can flushing be held off until either (1) blank line is printed,
(2) the 8k buffer fills, or (3) send_http_header is called?
1) is relevant only for handler that print headers, rather than set them
2) absolutely not, what if you want to flush data before?
3) send_http_header doesn't exist in Apache2/mod_perl2
I suppose that we could prevent flushing in the case the handler is
configured to parse headers. Does it make sense?
No. Could you explain your reasoning.
Only in the case that your handler is configured with:
PerlOptions +ParseHeaders
*and*
it prints headers ala:
print Content-type:
In all other cases where headers are set via the API, e.g. $r-content_type,
$r-headers_out, etc, there is no such a thing as the handler has send an
empty line signaling the end of sending headers, because it never sends any
headers at all, but uses api to set them.
Are we on the same page now?
mod_cgi uses spilling buckets, each of size 8K, to buffer script output,
including during the header scan, while mod_perl seems to scan the
headers
when the first 8K buffer is either filled or flushed.
I don't think this is related to the issue in question. Since the
problem is what to do on flush.
Also we might have a problem if the headers to parse are bigger than
the size of the buffer (8k). Do you think someone will ever need to
send headers bigger than 8k?
Yes, I mentioned the buffer size in case your objection to my proposal
to wait until end of headers was seen was based on the possiblity of
more than 8k of headers.
Again, the concept of the end of headers exists only in certain cases.
With the current mp2 code, if you decide to
wait for the end of headers before doing cgi parsing and flushing then
the code is assuming that either the headers are less than 8k or that any
Status header is in the first 8k. Otherwise the code would have to
be re-written to use continuous (spilling and merging) buffer buckets
like mod_cgi. It can hold off on flushing indefinitely.
That approach will break this handler:
sub handler {
my $r = shift;
$r-content_type('text/plain');
$r-rflush; # send something to the client immediately
long_job();
return Apache::OK
}
However notice that it doesn't have to set content_type() because some earlier
handler could have done that and that should work as well:
sub handler {
my $r = shift;
$r-rflush; # send something to the client immediately
long_job();
return Apache::OK
}
So as you can see, this handler doesn't tell us when it's done with headers.
OK, you may say that that previous handler should have marked the end of
headers settings, but that would be wrong if the response handler wants to set
other headers as well.
Do we now agree that the event of end of sending headers is possible only in
the case explained at the top?
__
Stas BekmanJAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide --- http://perl.apache.org
mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
Re: [mp2] CGI redirects incorrectly handled?
Mark James wrote: I'm having CGI redirect problems mp2 (cvs). Instead of being redirected to the proper web page, I'm sometimes getting a 302 Moved page containing a link to the correct URL. Damn this was a hard bug to track down. The cause of the problem was my perl code calling flush.pl and flushing STDOUT at a point prior to it printing the response headers. Under mp2, flushing STDOUT calls mpxs_output_flush in xs/Apache/RequestIO/Apache__RequestIO.h, which in turn calls ap_rflush, which triggers creation of the HTTP header, which at this stage, prior to my script printing its 302 header, uses a 200 OK status. So instead of a proper redirect being sent back to the browser, a normal web page with an embedded 302 link is sent. Everything seems to work if the ap_rflush call is removed from mpxs_output_flush, but I don't know if this is the proper way to fix it. Mark
Re: [mp2] CGI redirects incorrectly handled?
The cause of the problem was my perl code calling flush.pl and flushing STDOUT at a point prior to it printing the response headers. Under mp2, flushing STDOUT calls mpxs_output_flush in xs/Apache/RequestIO/Apache__RequestIO.h, which in turn calls ap_rflush, which triggers creation of the HTTP header, which at this stage, prior to my script printing its 302 header, uses a 200 OK status. Meaning no offence to the mp2 developpers, I find this observed behaviour inappropriate - I recently have to develop a reverse-proxy and got bitten by undocumented semantics of this sort every so often, I had to resort to reading the source with pencil paper like the original poster apparently did. What is the architectural justification for not choosing one of those two behaviours about header output, and erring on the middle side: * headers are out-of-band, and the first call to print() prepends whatever headers were set using the appropriate API (e.g. print_header() should have no effect afterwards, or maybe should set HTTP/1.1 trailers); * headers are regular flow, and Apache / mp2 never tries to add its own ones (almost impossible to ensure under Apache / mp1). Thanks for any insight on this topic - maybe there is a FAQ somewhere about MP2 architecture ? -- Dominique QUATRAVAUX Ingénieur développeur senior 01 44 42 00 08 IDEALX
Re: [mp2] CGI redirects incorrectly handled?
Mark James wrote: Mark James wrote: I'm having CGI redirect problems mp2 (cvs). Instead of being redirected to the proper web page, I'm sometimes getting a 302 Moved page containing a link to the correct URL. Damn this was a hard bug to track down. The cause of the problem was my perl code calling flush.pl and flushing STDOUT at a point prior to it printing the response headers. Under mp2, flushing STDOUT calls mpxs_output_flush in xs/Apache/RequestIO/Apache__RequestIO.h, which in turn calls ap_rflush, which triggers creation of the HTTP header, which at this stage, prior to my script printing its 302 header, uses a 200 OK status. So instead of a proper redirect being sent back to the browser, a normal web page with an embedded 302 link is sent. Hmm, why do you flush? The way Apache2 is designed is that the moment you send anything down the filter chain, the headers are generated, because they have to be sent before any data goes out. However mod_perl has an internal buffer and it won't flush the data before it gets full or the code tells it to flush using $r-rflush. If $|==0, then the buffer is not used and the data is flushed on every print. Everything seems to work if the ap_rflush call is removed from mpxs_output_flush, but I don't know if this is the proper way to fix it. No, this is not a proper way to fix it. Otherwise those who want to flush their output won't be able to do so. __ Stas BekmanJAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide --- http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com
Re: [mp2] CGI redirects incorrectly handled?
[EMAIL PROTECTED] wrote: The cause of the problem was my perl code calling flush.pl and flushing STDOUT at a point prior to it printing the response headers. Under mp2, flushing STDOUT calls mpxs_output_flush in xs/Apache/RequestIO/Apache__RequestIO.h, which in turn calls ap_rflush, which triggers creation of the HTTP header, which at this stage, prior to my script printing its 302 header, uses a 200 OK status. Meaning no offence to the mp2 developpers, I find this observed behaviour inappropriate - I recently have to develop a reverse-proxy and got bitten by undocumented semantics of this sort every so often, I had to resort to reading the source with pencil paper like the original poster apparently did. I think you are confusing mp2 design with Apache2 design. This is how Apache2 works, the main reason I believe to accomodate the filtering mechanism. As for undocumented behavior, you are welcome to submit documentation patches or wait till someone will write them. What is the architectural justification for not choosing one of those two behaviours about header output, and erring on the middle side: * headers are out-of-band, and the first call to print() prepends whatever headers were set using the appropriate API (e.g. print_header() should have no effect afterwards, or maybe should set HTTP/1.1 trailers); That's exactly how it works. The first print/puts/printf/rflush causes the headers to be sent (assuming that STDOUT is unbuffered), using whatever headers were set so far. Do I miss something here? This is 1:1 mapping to Apache behavior. The only difference is the mod_perl internal STDOUT buffer used for buffered STDOUT. * headers are regular flow, and Apache / mp2 never tries to add its own ones (almost impossible to ensure under Apache / mp1). Thanks for any insight on this topic - maybe there is a FAQ somewhere about MP2 architecture ? There are Doug's architecture notes online, I did some changes to them to bring things to the current state of things. They are certainly could have some more work. Patches are welcome. __ Stas BekmanJAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide --- http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com
Re: [mp2] CGI redirects incorrectly handled?
Stas Bekman wrote: Mark James wrote: The cause of the problem was my perl code calling flush.pl and flushing STDOUT at a point prior to it printing the response headers. Hmm, why do you flush? STDOUT is flushed prior to a fork to exec an external binary (rcs). The child is closing STDOUT and then redirecting it into a pipe to the parent. I didn't write this part of the code, but the comment on the flushing is: # flush now, lest data in a buffer get flushed on close() in every stinking # child process. The code for the forking is: bulletproof fork from camel book, 2ed, page 167 If necessary I can propose a patch to this perl package to make the flushing conditional on not running under mod_perl. The way Apache2 is designed is that the moment you send anything down the filter chain, the headers are generated, because they have to be sent before any data goes out. However mod_perl has an internal buffer and it won't flush the data before it gets full or the code tells it to flush using $r-rflush. If $|==0, then the buffer is not used and the data is flushed on every print. I see. But why is there no problem when using mod_cgi? Everything seems to work if the ap_rflush call is removed from mpxs_output_flush, but I don't know if this is the proper way to fix it. No, this is not a proper way to fix it. Otherwise those who want to flush their output won't be able to do so. Why would a perl handler script want to flush data down the filter chain before it had finished writing all of it? Mark
Re: [mp2] CGI redirects incorrectly handled?
Mark James wrote: Stas Bekman wrote: Mark James wrote: The cause of the problem was my perl code calling flush.pl and flushing STDOUT at a point prior to it printing the response headers. Hmm, why do you flush? STDOUT is flushed prior to a fork to exec an external binary (rcs). The child is closing STDOUT and then redirecting it into a pipe to the parent. I didn't write this part of the code, but the comment on the flushing is: # flush now, lest data in a buffer get flushed on close() in every stinking # child process. The code for the forking is: bulletproof fork from camel book, 2ed, page 167 If necessary I can propose a patch to this perl package to make the flushing conditional on not running under mod_perl. I understand the cause. But I hope that you agree with me that this is an application's problem. If you haven't sent anything to STDOUT yet, don't flush. And if this is not under your control, reopen STDOUT to /dev/null before you call that piece of code, that flushes and then re-tie STDOUT again. (See t/response/TestModperl/request_rec_tie_api.pm) Technically it's possible to add a flag in mod_perl 2.0 to ignore any flush attempts, if no data data was printed yet. However, this could become an undesirable behavior for someone who wants to send a flush before any data is sent. In your case, you can work around the problem, in the case of a person who wants the other behavior, there is no workaround. So I suggest that we keep the mp behavior generic and not create special cases we may regret about later on. The way Apache2 is designed is that the moment you send anything down the filter chain, the headers are generated, because they have to be sent before any data goes out. However mod_perl has an internal buffer and it won't flush the data before it gets full or the code tells it to flush using $r-rflush. If $|==0, then the buffer is not used and the data is flushed on every print. I see. But why is there no problem when using mod_cgi? That's an interesting question. mod_cgi is a generic handler, which can run applications written in any language. Therefore it has no clue of what flush is. It simply creates a pipe to the application, and expects the headers headers followed by the data. In your case, when cgi script flushes STDOUT, nothing happens at all, because there is no data to flush. So mod_cgi gets the headers and the data and all is cool. When the same code is run under mod_perl, flush generates a special bucket which is sent out to the filters chain, and since no headers are generated yet, they get generated and sent out. As I wrote this, I'm actually starting to think that it's Apache who should ignore the flush bucket if it had seen no other data so far, and not generate any headers till it actually sees the real data. Everything seems to work if the ap_rflush call is removed from mpxs_output_flush, but I don't know if this is the proper way to fix it. No, this is not a proper way to fix it. Otherwise those who want to flush their output won't be able to do so. Why would a perl handler script want to flush data down the filter chain before it had finished writing all of it? Here is an example: You have a long running process, you want the headers to be sent immediately, but the data won't follow for a while. So you create the headers, do $r-rflush, and later on send the data. __ Stas BekmanJAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide --- http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com
Re: [mp2] CGI redirects incorrectly handled?
Stas Bekman wrote: As I wrote this, I'm actually starting to think that it's Apache who should ignore the flush bucket if it had seen no other data so far, and not generate any headers till it actually sees the real data. And I went to produce a patch in http_filter, I figured that that would be wrong for the same reason, mod_perl shouldn't handle that as a special case, since that behavior might be a desired one. Another workaround for your problem could be a custom output filter, that yanks any bucket brigades including only the flush bucket, if it had not seen any real data buckets yet. __ Stas BekmanJAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide --- http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com
Re: [mp2] CGI redirects incorrectly handled?
Mark James wrote: Stas Bekman wrote: Can you send a short script (removing all the irrelevant bits) that we can reproduce the problem with? Made a script that generated the same POST request and same redirect as the problem code. The problem was not reproduced! The only difference I can see between working POSTs (both those in my package code and the one in the test script) and the failing one (a particular one in my package) is in the distribution of the data across the TCP packets that carry the POST. [snippet the packet dumps] Could mod_perl, with its persistent buffer, be tripping up on this? I'm trying now to trace the data through the mp2 code. -- Mark mod_perl 2.0 is buffering only the content (response body, not the headers) (assuming that you aren't installing any output filters), so it's one of the apache core output filters, that decides how to split the data. a normal output filter list ends with: byterange(0x8841110): ctx=0x0, r=0x88402d0, c=0x883a390 content_length(0x8841128): ctx=0x0, r=0x88402d0, c=0x883a390 http_header(0x8841140): ctx=0x0, r=0x88402d0, c=0x883a390 core(0x883a760): ctx=0x883a738, r=0x0, c=0x883a390 you can dump this, using the dump_filters macro, which you can load by running gdb source /path/to/httpd-2.0/.gdbinit the 'core' filter is in httpd-2.0/src/core.c: core_output_filter(...) __ Stas BekmanJAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide --- http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com
Re: [mp2] CGI redirects incorrectly handled?
Stas Bekman wrote:
Can you send a short script (removing all the irrelevant bits) that we
can reproduce the problem with?
Made a script that generated the same POST request and same
redirect as the problem code. The problem was not reproduced!
The only difference I can see between working POSTs (both those
in my package code and the one in the test script) and the failing
one (a particular one in my package) is in the distribution of the
data across the TCP packets that carry the POST.
Packet 1 always has the POST request and a set of headers.
When it works (a 302 is sent), Packet 2 has the Content-Type
header, Packet 3 has the Content-Length header, and Packet 4
has the POSTed variables:
PACKET 2:
0x 4500 0065 f659 4000 4006 4993 9084 ecce[EMAIL PROTECTED]@.I.
0x0010 9084 ecce 9ea1 0050 482e 99c3 4828 27ae...PH...H('.
0x0020 8018 7fff cd14 0101 080a 00a9 6185..a.
0x0030 00a9 6185 436f 6e74 656e 742d 5479 7065..a.Content-Type
0x0040 3a20 6170 706c 6963 6174 696f 6e2f 782d:.application/x-
0x0050 772d 666f 726d 2d75 726c 656e 636fwww-form-urlenco
0x0060 6465 640d 0a ded..
PACKET 3:
0x 4500 004b f65a 4000 4006 49ac 9084 ecce[EMAIL PROTECTED]@.I.
0x0010 9084 ecce 9ea1 0050 482e 99f4 4828 27ae...PH...H('.
0x0020 8018 7fff d88d 0101 080a 00a9 6185..a.
0x0030 00a9 6185 436f 6e74 656e 742d 4c65 6e67..a.Content-Leng
0x0040 7468 3a20 3334 320d 0a0d 0ath:.342
PACKET 4:
0x 4500 018a f65b 4000 4006 486c 9084 ecce[EMAIL PROTECTED]@.Hl
0x0010 9084 ecce 9ea1 0050 482e 9a0b 4828 27ae...PH...H('.
0x0020 8018 7fff 945c 0101 080a 00a9 6185.\a.
0x0030 00a9 6185 5f70 6173 735f 6964 3d75 7365..a._pass_id=use
0x0040 7233 2534 306d 616b 6574 6865 6361 7365r3%40makethecase
0x0050 2e6e 6574 265f 7061 7373 5f70 6173 733d.net_pass_pass=
[rest of packet cut]
But when it fails (a 200 with a 302 link is sent),
all is in the one packet:
0x 4500 01af 1d11 4000 4006 2192 9084 ecce[EMAIL PROTECTED]@.!.
0x0010 9084 ecce 9faf 0050 f769 cb03 f764 9ec7...P.i...d..
0x0020 8018 bb9e fe65 0101 080a 00ad a4b7.e..
0x0030 00ad a4b7 436f 6e74 656e 742d 5479 7065Content-Type
0x0040 3a20 6170 706c 6963 6174 696f 6e2f 782d:.application/x-
0x0050 772d 666f 726d 2d75 726c 656e 636fwww-form-urlenco
0x0060 6465 640d 0a43 6f6e 7465 6e74 2d4c 656eded..Content-Len
0x0070 6774 683a 2033 3037 0d0a 0d0a 5f70 6173gth:.307_pas
0x0080 735f 6964 3d75 7365 7233 2534 306d 616bs_id=user3%40mak
0x0090 6574 6865 6361 7365 2e6e 6574 265f 7061ethecase.net_pa
0x00a0 7373 5f70 6173 733dss_pass=
[rest of packet cut]
Could mod_perl, with its persistent buffer, be tripping up on this?
I'm trying now to trace the data through the mp2 code. -- Mark
Re: [mp2] CGI redirects incorrectly handled?
How are you telling the server to redirect? You do know it's different
from mp1, right?
In mp2 you need to do:
my $location = 'http://foo.bar.baz';
$r-headers_out-{'Location'} = $location;
# Or use $r-err_headers_out-{'Location'} which you will have
# to do with any other headers you want to have sent with the
# redirect, such as cookies
return Apache::HTTP_MOVED_TEMPORARILY;
# Apache::REDIRECT still supported, this is the correct
# constant though.
On Wed, 5 Mar 2003, Mark James wrote:
I'm having CGI redirect problems mp2 (cvs).
How are you telling the server to redirect? You do know it's different
from mp1, right?
In mp2 you need to do:
my $location = 'http://foo.bar.baz';
$r-headers_out-{'Location'} = $location;
# Or use $r-err_headers_out-{'Location'} which you will have
# to do with any other headers you want to have sent with the
# redirect, such as cookies
return Apache::HTTP_MOVED_TEMPORARILY;
# Apache::REDIRECT still supported, this is the correct
# constant though.
Now that I think about it, maybe you're using CGI.pm to do your redirect?
If so, maybe the code in CGI.pm has not been correctly updated?
Hope this helps.
- nick
--
Nick Tonkin {|8^)
Re: [mp2] CGI redirects incorrectly handled?
On Thu, 6 Mar 2003, Mark James wrote:
Nick Tonkin wrote:
Now that I think about it, maybe you're using CGI.pm to do your redirect?
If so, maybe the code in CGI.pm has not been correctly updated?
Yes Nick, I'm using CGI.pm version 2.91 (the latest). Its redirect code
sends a Status: 302 Moved.
That wouldn't be the problem, if there is one. The problem would be in how
CGI.pm sets the Location header, noting the differences in syntax I
pointed out earlier.
You might try one of the CGI mailing lists to see if anyone there knows
whether the code is compliant with what I posted before. The documentation
is at http://xrl.us/dfb
Otherwise, try setting the redirect location manually as I showed you and
see if the problem persists.
- nick
--
Nick Tonkin {|8^)
Re: [mp2] CGI redirects incorrectly handled?
Mark James wrote:
I'm having CGI redirect problems mp2 (cvs).
Instead of being redirected to the proper web page, I'm sometimes
getting a 302 Moved page containing a link to the correct URL.
Seems to be related to the following code in modperl_cgi.c:
if (location (location[0] == '/') (r-status == 200)) {
r-method = apr_pstrdup(r-pool, GET);
r-method_number = M_GET;
The Location field I'm redirecting to is a fully-qualified URL,
starting with http://, but still at the local server. A debug
put in above this code confirms that location is set to a
string starting with http.
Why is the test for location[0] == '/' there? Which section
of code is usually responsible for stripping off the server
part of the address if it is local?
as the comment just above this line says, that code was copy-n-pasted from
mod_cgi. Can you reproduce the same problem while running a cgi script?
Also could it be that it has to do with the recent change, I've applied which
was already reported by Beau as broken. May be your headers don't get parsed
What happens if you do:
Index: src/modules/perl/modperl_filter.c
===
RCS file: /home/cvs/modperl-2.0/src/modules/perl/modperl_filter.c,v
retrieving revision 1.54
diff -u -r1.54 modperl_filter.c
--- src/modules/perl/modperl_filter.c 3 Mar 2003 03:39:06 - 1.54
+++ src/modules/perl/modperl_filter.c 5 Mar 2003 23:15:44 -
@@ -55,7 +55,7 @@
apr_bucket *bucket;
const char *work_buf = buf;
-if (wb-header_parse !wb-r-content_type) {
+if (wb-header_parse) {
request_rec *r = wb-r;
const char *bodytext = NULL;
int status;
__
Stas BekmanJAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide --- http://perl.apache.org
mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
Re: [mp2] CGI redirects incorrectly handled?
Stas Bekman wrote: Mark James wrote: I'm having CGI redirect problems mp2 (cvs). as the comment just above this line says, that code was copy-n-pasted from mod_cgi. Can you reproduce the same problem while running a cgi script? No, them problem only manifests under mod_perl (2, haven't used 1). Also could it be that it has to do with the recent change, I've applied which was already reported by Beau as broken. May be your headers don't get parsed What happens if you do: [patch] Applied the patch, but the problem still occurred. No change also when I commented out the location[0]=='/' test. The redirect header being printed by my perl script is: Server: Apache/2.0.44 (Unix) mod_perl/1.99_09-dev Perl/v5.8.0 Status: 302 Moved Date: Thu, 06 Mar 2003 01:10:54 GMT Location: http://makethecase.net/db?auth=ckffb2a5c44ee0editCmds=compactfile=62 Which is returned as a 302 link page. This is a redirect response to a POST. Strangely, another redirect, with header: Server: Apache/2.0.44 (Unix) mod_perl/1.99_09-dev Perl/v5.8.0 Status: 302 Moved Date: Thu, 06 Mar 2003 01:15:54 GMT Location: http://makethecase.net/db?_reason=6%20Case1Pro_restart=editPartcheckSequenceNumber=60cmd=authenticateeditCmds=compactfile=62partnum= works just fine. This is a redirect after a GET. Mark
Re: [mp2] CGI redirects incorrectly handled?
Mark James wrote: Stas Bekman wrote: Mark James wrote: I'm having CGI redirect problems mp2 (cvs). as the comment just above this line says, that code was copy-n-pasted from mod_cgi. Can you reproduce the same problem while running a cgi script? No, them problem only manifests under mod_perl (2, haven't used 1). Sorry, I'm not following your comment. I've suggested to test with mod_cgi (under Apache2), since mod_perl mimics mod_cgi's behavior here. Also could it be that it has to do with the recent change, I've applied which was already reported by Beau as broken. May be your headers don't get parsed What happens if you do: [patch] Applied the patch, but the problem still occurred. No change also when I commented out the location[0]=='/' test. The redirect header being printed by my perl script is: Server: Apache/2.0.44 (Unix) mod_perl/1.99_09-dev Perl/v5.8.0 Status: 302 Moved Date: Thu, 06 Mar 2003 01:10:54 GMT Location: http://makethecase.net/db?auth=ckffb2a5c44ee0editCmds=compactfile=62 Which is returned as a 302 link page. This is a redirect response to a POST. Strangely, another redirect, with header: Server: Apache/2.0.44 (Unix) mod_perl/1.99_09-dev Perl/v5.8.0 Status: 302 Moved Date: Thu, 06 Mar 2003 01:15:54 GMT Location: http://makethecase.net/db?_reason=6%20Case1Pro_restart=editPartcheckSequenceNumber=60cmd=authenticateeditCmds=compactfile=62partnum= works just fine. This is a redirect after a GET. Should POST-redirect return 307? http://ppewww.ph.gla.ac.uk/~flavell/www/post-redirect.html http://rfc.net/rfc2616.html#s10.3.8 __ Stas BekmanJAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide --- http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com
Re: [mp2] CGI redirects incorrectly handled?
Mark James wrote: 303 See Other is the correct post-POST redirect response: http://rfc.net/rfc2616.html#s10.3.4 which your first link suggests works in all browsers. Well, taking a closer look, 303 doesn't work in Netscape 3 or 4. CGI.pm always returns a 302, though, if necessary, I can edit its reply in my script before printing it. I'll give this a go. This didn't work. Got a 303 link page instead of a 302 one.
