Re: ENC: [mp2.0] problems when built modperl
Rosane Novello wrote: When I try to build I receive the error above. Someone can help me? C:\mod_perl-1.99_08perl Makefile.PL MP_AP_PREFIX=C:\Apache Group\Apache2 Use of uninitialized value in concatenation (.) or string at lib/ModPerl/BuildOptions.pm line 89, DATA line 18. A comment in BuildOptions.pm states MP_AP_PREFIX may not contain spaces.
Re: [mp2] CGI redirects incorrectly handled?
Stas Bekman wrote:
So can flushing be held off until either (1) blank line is printed,
(2) the 8k buffer fills, or (3) send_http_header is called?
1) is relevant only for handler that print headers, rather than set them
2) absolutely not, what if you want to flush data before?
3) send_http_header doesn't exist in Apache2/mod_perl2
I didn't realise that mp2 doesn't use send_http_header. That explains
the appearance of wb-r-content_type in the mp2 code. So is it
true that if headers are sent using the API then no output filtering
and transmission occurs until the 8k buffer is either filled or flushed
(explcitly or after exit)?
Only in the case that your handler is configured with:
PerlOptions +ParseHeaders
*and*
it prints headers ala:
print Content-type:
In all other cases where headers are set via the API, e.g.
$r-content_type, $r-headers_out, etc, there is no such a thing as the
handler has send an empty line signaling the end of sending headers,
because it never sends any headers at all, but uses api to set them.
Is +ParseHeaders always indicative of explicit header printing, or
can it also be set when using the API? If the former, then yes, if
+ParseHeaders is set flushing could be held off until a blank line is seen.
With the current mp2 code, if you decide to
wait for the end of headers before doing cgi parsing and flushing then
the code is assuming that either the headers are less than 8k or that any
Status header is in the first 8k. Otherwise the code would have to
be re-written to use continuous (spilling and merging) buffer buckets
like mod_cgi. It can hold off on flushing indefinitely.
That approach will break this handler:
sub handler {
my $r = shift;
$r-content_type('text/plain');
$r-rflush; # send something to the client immediately
long_job();
return Apache::OK
}
However notice that it doesn't have to set content_type() because some
earlier handler could have done that and that should work as well:
sub handler {
my $r = shift;
$r-rflush; # send something to the client immediately
long_job();
return Apache::OK
}
So as you can see, this handler doesn't tell us when it's done with
headers.
OK, you may say that that previous handler should have marked the end of
headers settings, but that would be wrong if the response handler wants
to set other headers as well.
Do we now agree that the event of end of sending headers is possible
only in the case explained at the top?
Yes, the key I was missing is that mp2 no longer uses send_http_header.
Can you just lock out flushing when nothing has been printed and
content_type is undefined? (You impliy above that the content_type
setting is persistent, so the script would have to undef it if necessary.)
Then all the user script has to do is to make sure any Status header is
either printed or set via headers_out in the first batch of
printing/setting code before flush is called (again).
Re: [mp2] CGI redirects incorrectly handled?
Stas Bekman wrote: Mark James wrote: STDOUT is flushed prior to a fork to exec an external binary (rcs). I understand the cause. But I hope that you agree with me that this is an application's problem. If you haven't sent anything to STDOUT yet, don't flush. And if this is not under your control, reopen STDOUT to /dev/null before you call that piece of code, that flushes and then re-tie STDOUT again. (See t/response/TestModperl/request_rec_tie_api.pm) I guess the best way to fix the problem in-application would be to either run nph, or do the /dev/null redirect you suggest. Interestingly, commenting out the pre-fork flush fixes the problem under mod_perl because close in mod_perl seems to be a no-op rather than a flush. If the close is also no problem under mod_cgi then is there any real need for such a pre-fork flush in my script? I see. But why is there no problem when using mod_cgi? That's an interesting question. mod_cgi is a generic handler, which can run applications written in any language. Therefore it has no clue of what flush is. It simply creates a pipe to the application, and expects the headers headers followed by the data. In your case, when cgi script flushes STDOUT, nothing happens at all, because there is no data to flush. So mod_cgi gets the headers and the data and all is cool When the same code is run under mod_perl, flush generates a special bucket which is sent out to the filters chain, and since no headers are generated yet, they get generated and sent out. Well, even under mod_cgi a program can still fflush or write. The difference between mod_cgi and mod_perl is that mod_cgi does not activate the filter brigade until it has read all the headers. Why would a perl handler script want to flush data down the filter chain before it had finished writing all of it? Here is an example: You have a long running process, you want the headers to be sent immediately, but the data won't follow for a while. So you create the headers, do $r-rflush, and later on send the data. OK. So would there be a problem if mod_perl waited for the blank line end of headers, or at least a Status header, before passing the buckets down the line, just like mod_cgi? mod_cgi uses spilling buckets, each of size 8K, to buffer script output, including during the header scan, while mod_perl seems to scan the headers when the first 8K buffer is either filled or flushed.
Re: [mp2] CGI redirects incorrectly handled?
Stas Bekman wrote: Since the mod_perl's internal STDOUT buffer isn't mangled if you re-tie it later, and it'll be always flushed at the end of the request, there is no *need* to flush on CLOSE. However in order to be consistent with perl fh close behavior, it probably needs to be changed to flush its buffer. What do you think? Dunno. But the problem I had would have been even harder to track down if commenting out the flush hadn't fixed it. The difference between mod_cgi and mod_perl is that mod_cgi does not activate the filter brigade until it has read all the headers. But in the case of mod_perl, this event is valid only for handlers which print their own headers, rather than using mod_perl API to set them. In the generic case, there is no way to tell whether a handler is going to set more headers or it has done with it. So can flushing be held off until either (1) blank line is printed, (2) the 8k buffer fills, or (3) send_http_header is called? I suppose that we could prevent flushing in the case the handler is configured to parse headers. Does it make sense? No. Could you explain your reasoning. mod_cgi uses spilling buckets, each of size 8K, to buffer script output, including during the header scan, while mod_perl seems to scan the headers when the first 8K buffer is either filled or flushed. I don't think this is related to the issue in question. Since the problem is what to do on flush. Also we might have a problem if the headers to parse are bigger than the size of the buffer (8k). Do you think someone will ever need to send headers bigger than 8k? Yes, I mentioned the buffer size in case your objection to my proposal to wait until end of headers was seen was based on the possiblity of more than 8k of headers. With the current mp2 code, if you decide to wait for the end of headers before doing cgi parsing and flushing then the code is assuming that either the headers are less than 8k or that any Status header is in the first 8k. Otherwise the code would have to be re-written to use continuous (spilling and merging) buffer buckets like mod_cgi. It can hold off on flushing indefinitely.
Re: [mp2] CGI redirects incorrectly handled?
Mark James wrote: I'm having CGI redirect problems mp2 (cvs). Instead of being redirected to the proper web page, I'm sometimes getting a 302 Moved page containing a link to the correct URL. Damn this was a hard bug to track down. The cause of the problem was my perl code calling flush.pl and flushing STDOUT at a point prior to it printing the response headers. Under mp2, flushing STDOUT calls mpxs_output_flush in xs/Apache/RequestIO/Apache__RequestIO.h, which in turn calls ap_rflush, which triggers creation of the HTTP header, which at this stage, prior to my script printing its 302 header, uses a 200 OK status. So instead of a proper redirect being sent back to the browser, a normal web page with an embedded 302 link is sent. Everything seems to work if the ap_rflush call is removed from mpxs_output_flush, but I don't know if this is the proper way to fix it. Mark
Re: [mp2] CGI redirects incorrectly handled?
Stas Bekman wrote: Mark James wrote: The cause of the problem was my perl code calling flush.pl and flushing STDOUT at a point prior to it printing the response headers. Hmm, why do you flush? STDOUT is flushed prior to a fork to exec an external binary (rcs). The child is closing STDOUT and then redirecting it into a pipe to the parent. I didn't write this part of the code, but the comment on the flushing is: # flush now, lest data in a buffer get flushed on close() in every stinking # child process. The code for the forking is: bulletproof fork from camel book, 2ed, page 167 If necessary I can propose a patch to this perl package to make the flushing conditional on not running under mod_perl. The way Apache2 is designed is that the moment you send anything down the filter chain, the headers are generated, because they have to be sent before any data goes out. However mod_perl has an internal buffer and it won't flush the data before it gets full or the code tells it to flush using $r-rflush. If $|==0, then the buffer is not used and the data is flushed on every print. I see. But why is there no problem when using mod_cgi? Everything seems to work if the ap_rflush call is removed from mpxs_output_flush, but I don't know if this is the proper way to fix it. No, this is not a proper way to fix it. Otherwise those who want to flush their output won't be able to do so. Why would a perl handler script want to flush data down the filter chain before it had finished writing all of it? Mark
Re: [mp2] CGI redirects incorrectly handled?
Stas Bekman wrote:
Can you send a short script (removing all the irrelevant bits) that we
can reproduce the problem with?
Made a script that generated the same POST request and same
redirect as the problem code. The problem was not reproduced!
The only difference I can see between working POSTs (both those
in my package code and the one in the test script) and the failing
one (a particular one in my package) is in the distribution of the
data across the TCP packets that carry the POST.
Packet 1 always has the POST request and a set of headers.
When it works (a 302 is sent), Packet 2 has the Content-Type
header, Packet 3 has the Content-Length header, and Packet 4
has the POSTed variables:
PACKET 2:
0x 4500 0065 f659 4000 4006 4993 9084 ecce[EMAIL PROTECTED]@.I.
0x0010 9084 ecce 9ea1 0050 482e 99c3 4828 27ae...PH...H('.
0x0020 8018 7fff cd14 0101 080a 00a9 6185..a.
0x0030 00a9 6185 436f 6e74 656e 742d 5479 7065..a.Content-Type
0x0040 3a20 6170 706c 6963 6174 696f 6e2f 782d:.application/x-
0x0050 772d 666f 726d 2d75 726c 656e 636fwww-form-urlenco
0x0060 6465 640d 0a ded..
PACKET 3:
0x 4500 004b f65a 4000 4006 49ac 9084 ecce[EMAIL PROTECTED]@.I.
0x0010 9084 ecce 9ea1 0050 482e 99f4 4828 27ae...PH...H('.
0x0020 8018 7fff d88d 0101 080a 00a9 6185..a.
0x0030 00a9 6185 436f 6e74 656e 742d 4c65 6e67..a.Content-Leng
0x0040 7468 3a20 3334 320d 0a0d 0ath:.342
PACKET 4:
0x 4500 018a f65b 4000 4006 486c 9084 ecce[EMAIL PROTECTED]@.Hl
0x0010 9084 ecce 9ea1 0050 482e 9a0b 4828 27ae...PH...H('.
0x0020 8018 7fff 945c 0101 080a 00a9 6185.\a.
0x0030 00a9 6185 5f70 6173 735f 6964 3d75 7365..a._pass_id=use
0x0040 7233 2534 306d 616b 6574 6865 6361 7365r3%40makethecase
0x0050 2e6e 6574 265f 7061 7373 5f70 6173 733d.net_pass_pass=
[rest of packet cut]
But when it fails (a 200 with a 302 link is sent),
all is in the one packet:
0x 4500 01af 1d11 4000 4006 2192 9084 ecce[EMAIL PROTECTED]@.!.
0x0010 9084 ecce 9faf 0050 f769 cb03 f764 9ec7...P.i...d..
0x0020 8018 bb9e fe65 0101 080a 00ad a4b7.e..
0x0030 00ad a4b7 436f 6e74 656e 742d 5479 7065Content-Type
0x0040 3a20 6170 706c 6963 6174 696f 6e2f 782d:.application/x-
0x0050 772d 666f 726d 2d75 726c 656e 636fwww-form-urlenco
0x0060 6465 640d 0a43 6f6e 7465 6e74 2d4c 656eded..Content-Len
0x0070 6774 683a 2033 3037 0d0a 0d0a 5f70 6173gth:.307_pas
0x0080 735f 6964 3d75 7365 7233 2534 306d 616bs_id=user3%40mak
0x0090 6574 6865 6361 7365 2e6e 6574 265f 7061ethecase.net_pa
0x00a0 7373 5f70 6173 733dss_pass=
[rest of packet cut]
Could mod_perl, with its persistent buffer, be tripping up on this?
I'm trying now to trace the data through the mp2 code. -- Mark
[mp2] CGI redirects incorrectly handled?
I'm having CGI redirect problems mp2 (cvs).
Instead of being redirected to the proper web page, I'm sometimes
getting a 302 Moved page containing a link to the correct URL.
Seems to be related to the following code in modperl_cgi.c:
if (location (location[0] == '/') (r-status == 200)) {
r-method = apr_pstrdup(r-pool, GET);
r-method_number = M_GET;
The Location field I'm redirecting to is a fully-qualified URL,
starting with http://, but still at the local server. A debug
put in above this code confirms that location is set to a
string starting with http.
Why is the test for location[0] == '/' there? Which section
of code is usually responsible for stripping off the server
part of the address if it is local?
Mark
Re: mp2 runs scripts as root?
Stas Bekman wrote:
...
Well, actually this is not the case. It behaves exactly the same as in
modperl 1.0. You can use Env::C module (available from CPAN) to easily
debug this:
...
In any case you probably want to rely on getpwuid($), rather than
$ENV{USER} if applicable.
Unfortunately ci (RCS) uses the USER variable to determine lock
permissions for edits.
But using your Env::C::setenv to set USER instead of assigning to
$ENV{'USER'} allowed me to remove the ENV assigments from start-up.pl.
This'll make my package more turn-key.
Thanks again -- Mark
Re: Reading an array from perl script
Devi .M wrote: Hello All, I have a perl script that would be running infinitely and updating an array by processing some data. Now I would like to read the array values that should not disturb this perl script. I thought to use the concept of shared variable and write a method in perl module which contains the array sharring with the infinitely running perl script and return me the updated array values. But don't know how to proceed with this. Could anyone help me out? Perhaps this IPC::Shareable - share Perl variables between processes package would be suitable: http://search.cpan.org/author/BSUGARS/IPC-Shareable-0.60/lib/IPC/Shareable.pm Mark
Re: [mp2] CGI redirects incorrectly handled?
Stas Bekman wrote: Mark James wrote: I'm having CGI redirect problems mp2 (cvs). as the comment just above this line says, that code was copy-n-pasted from mod_cgi. Can you reproduce the same problem while running a cgi script? No, them problem only manifests under mod_perl (2, haven't used 1). Also could it be that it has to do with the recent change, I've applied which was already reported by Beau as broken. May be your headers don't get parsed What happens if you do: [patch] Applied the patch, but the problem still occurred. No change also when I commented out the location[0]=='/' test. The redirect header being printed by my perl script is: Server: Apache/2.0.44 (Unix) mod_perl/1.99_09-dev Perl/v5.8.0 Status: 302 Moved Date: Thu, 06 Mar 2003 01:10:54 GMT Location: http://makethecase.net/db?auth=ckffb2a5c44ee0editCmds=compactfile=62 Which is returned as a 302 link page. This is a redirect response to a POST. Strangely, another redirect, with header: Server: Apache/2.0.44 (Unix) mod_perl/1.99_09-dev Perl/v5.8.0 Status: 302 Moved Date: Thu, 06 Mar 2003 01:15:54 GMT Location: http://makethecase.net/db?_reason=6%20Case1Pro_restart=editPartcheckSequenceNumber=60cmd=authenticateeditCmds=compactfile=62partnum= works just fine. This is a redirect after a GET. Mark
Re: [mp2] CGI redirects incorrectly handled?
Mark James wrote: 303 See Other is the correct post-POST redirect response: http://rfc.net/rfc2616.html#s10.3.4 which your first link suggests works in all browsers. Well, taking a closer look, 303 doesn't work in Netscape 3 or 4. CGI.pm always returns a 302, though, if necessary, I can edit its reply in my script before printing it. I'll give this a go. This didn't work. Got a 303 link page instead of a 302 one.
Scripts and HTML docs in the same directory (+ modperl newbie experiences)
Hello All, Took me a day, but I think I've finally been able to move my scripts from plain cgi perl to mod_perl. The extensive documention on perl.apache.org was invaluable, though I have some comments below. One question: Prior to using mod_perl I was able to have unsuffixed scripts and .html files residing together in the server root directory by using Apache's Files directive to force the scripts to be executed: e.g. Files db ForceType application/x-httpd-cgi /Files I can't seem to do this with mod_perl because I now have to use a SetHandler directive on the directory, so Apache now tries to execute the HTML files. I've tried forcing the scripts to type application/x-httpd-perl, but this doesn't work. I'd like to keep both the scripts and HTML docs in the root directory so that URLs are as short as possible, but can I do this with mod_perl without having to use URL re-writing? Also having a problem with the OPEN method in Apache::RequestRec not being found, but a reading of the list archives suggests that an upgrade to the CVS version of mod_perl-2 will fix this. Finally, some comments on mod_perl installation: 1. In http://perl.apache.org/docs/1.0/guide/getwet.html , use of x.x.x for both the Apache and mod_perl version numbers made me think that the version numbers had to be matched. Maybe y.y.y should be used for one. It also should be stated in this section that one has to use mod_perl-2.x if one is running Apache 2.y. 2. When installing mod_perl-2 I had the problem others have had with failed perlio tests. Looks to be due to the test files being created using the real UID rather than the effective UID. 3. In the configuration section of the 2.0 docs (http://perl.apache.org/docs/2.0/user/intro/start_fast.html#toc_Configuration) it neglects to state the need to issue a directive for the mod_perl handler one is going to use, e.g. PerlModule ModPerl::Registry. Thanks to all mod_perl coders!
Re: mp2 runs scripts as root?
Stas Bekman wrote:
Mark James wrote:
Some of my scripts break when running under mp2 (cvs) because the UID
is set as root rather than the Apache user (which for me is web).
The problem manifests with RCS file locking. Is there some switch
to set so that I can run scripts as web?
Eh? Are you talking about 'make test' or installed mod_perl? If the
former, use the latest cvs where this should work if the latter, modify
httpd.conf.
The latter. Turned out to be a caused by the perl ENV not being
propogated to forked programs:
http://perl.apache.org/docs/2.0/user/troubleshooting/troubleshooting.html#C_Libraries_Don_t_See_C__ENV__Entries_Set_by_Perl_Code
Fixed by adding:
$ENV{'USER'} = 'web';
$ENV{'LOGNAME'} = 'web';
to the start-up script.
Thanks for the reply -- Mark
mp2 runs scripts as root?
Some of my scripts break when running under mp2 (cvs) because the UID is set as root rather than the Apache user (which for me is web). The problem manifests with RCS file locking. Is there some switch to set so that I can run scripts as web?
Re: Scripts and HTML docs in the same directory (+ modperl newbieexperiences)
Stas Bekman wrote: Mark James wrote: 1. In http://perl.apache.org/docs/1.0/guide/getwet.html , use of x.x.x for both the Apache and mod_perl version numbers made me think that the version numbers had to be matched. Maybe y.y.y should be used for one. Please get used to x.x.x meaning any. Otherwise we would need to remember to use z.z.z. for php plugs in and f.f.f. when openssl is added, etc... hope you get the idea. When they're discussed in the same sentence, and when building one requires linking to the installation or source directory of the other, I think a different variable helps. 2. In the configuration section of the 2.0 docs (http://perl.apache.org/docs/2.0/user/intro/start_fast.html#toc_Configuration) it neglects to state the need to issue a directive for the mod_perl handler one is going to use, e.g. PerlModule ModPerl::Registry, though it is covered in the configuration docs (including the startup-file option). you mean preloading the module? That's not necessarily in mp2, though advisable for performance reasons. In mp2, you can say: PerlResponseHandler ModPerl::Registry without: PerlModule ModPerl::Registry Well I just commented out use ModPerl::Registry () in my startup script and it still worked. But earlier I had found that adding PerlModule ModPerl::Registry the http.conf was the key to getting rid of the rash of error messages I was getting on server start-up. It must have been a manifestation of some other problem, perhaps with mod_perl-1.99_08 (now using the CVS version to fix a missing OPEN in Apache::RequestRec, and to avoid the failed perlio tests), or with an older CGI.pm (found out late that CGI=2.89 was needed). you can also use the syntactic sugar to preload modules, by simply stating at the beginning of your mod_perl configuration 'PerlOptions +Autoload'. See http://perl.apache.org/docs/2.0/user/config/config.html#C_AutoLoad_ or using + before the handler name: PerlResponseHandler +ModPerl::Registry OK, so 2.0 is not like 1.0 where PerlModule acts like use() (http://perl.apache.org/docs/1.0/guide/config.html#PerlModule_and_PerlRequire_Directives), but is more like @INC manipulation; and these handler autoload directives are an alternative to use-ing them in a start-up script. My start-up script is very long because it calls use for just about every package in an extensive package set. I suppose an import function could be created in a master package of package suite that when called require-ed all the associated packages, so that mod_perl can have the entire suite pre-loaded prior to forking through just one line in the start-up script. Thank you Stas for your prompt help. Mark
Scripts and HTML docs in the same directory (+ modperl newbie experiences)
Hello All, Took me a day, but I think I've finally been able to move my scripts from plain cgi perl to mod_perl2. The extensive documention on perl.apache.org was invaluable, though I have some comments below. One question: Prior to using mod_perl I was able to have unsuffixed scripts and .html files residing together in the server root directory by using Apache's Files directive to force the scripts to be executed: e.g. Files db ForceType application/x-httpd-cgi /Files I can't seem to do this with mod_perl because I now have to use a SetHandler directive on the directory, so Apache now executes the HTML files. I've tried forcing the scripts to type application/x-httpd-perl, but this doesn't work. I'd like to keep both the scripts and HTML docs in the root directory so that URLs are as short as possible, but can I do this with mod_perl without having to use URL re-writing? Finally, some comments on mod_perl installation: 1. In http://perl.apache.org/docs/1.0/guide/getwet.html , use of x.x.x for both the Apache and mod_perl version numbers made me think that the version numbers had to be matched. Maybe y.y.y should be used for one. It also should be stated in this section that one has to use mod_perl-2.x if one is running Apache 2.y. 2. In the configuration section of the 2.0 docs (http://perl.apache.org/docs/2.0/user/intro/start_fast.html#toc_Configuration) it neglects to state the need to issue a directive for the mod_perl handler one is going to use, e.g. PerlModule ModPerl::Registry, though it is covered in the configuration docs (including the startup-file option). Thanks to all mod_perl coders!
Re: Scripts and HTML docs in the same directory (+ modperl newbieexperiences)
Perrin Harkins wrote: You should be able to do the SetHandler inside a Files directive just like you did with ForceType. Have you tried that? Thanks Perrin, that worked. I didn't think SetHandler directives were allowed in Files sections, because it's not listed in the SetHandler docs (http://httpd.apache.org/docs-2.0/mod/core.html#sethandler), unlike ForceType docs (http://httpd.apache.org/docs-2.0/mod/core.html#forcetype) Stas Bekman wrote: Set the normal behavior per dir/location and then override for specific files Stas' solution would therefore work as well. Mark
