Re: Server questions
Hi there, On Fri, 7 Mar 2003, Michael Hyman wrote: I am not familiar with clustering Would you run a mod_perl based web site on a cluster? If the performance and the money for the hardware are issues then perhaps before you buy you should spend some time looking into things like alternative system architectures, software packages, development methods, timescales and *those* costs... There's a lot more to it than what's the fastest machine. Apache isn't the fastest Web server on the planet and coding it in Perl isn't the fastest way of implementing an algorithm. Asking for data from Oracle won't usually be the fastest way to get hold of it - especially if the machine running Oracle is remote. On the same hardware, you might get ten times the performance from a well-tuned proxy server setup than you can from a single mod_perl server. And you might not. Put all those things into a system and the difference between Solaris and Linux or between PC and Sparc may well be lost in the noise. And if you don't have a pretty good idea of where you're going with it all before you set out, then you might not get there. Have you any metrics for the kinds of loads you expect to meet, and the parts of the system which will use most resources to sustain them? Do you know what performance you can expect from the database under the expected load conditions? Is any of this under your control? 73, Ged.
Re: [mp2] Latest mod_perl funny test results
On 8 Mar 2003 at 20:56, Stas Bekman wrote: Stas Bekman wrote: Beau E. Cox wrote: Hi - I just upgraded to Apache 2.0.44 (source) and mod_perl-1.99-09-dev (CVS source) on my SuSE 8.1 test server. All test scripts completed OK. I am omitting the details of my configuration - will supply them if anyone thinks this problem is related to them. As I have with prior installations, I am tip-toeing through the tests until my server is completely up to date (+mason, +mod_ssl, +lots of vhosts, etc.). [...] And wrote the test script in /srv/www/perl/rocks.html (with the proper permissions/ownership): #!/usr/bin/perl print Content-type: text/html\n\n; print h3mod_perl 2.0 rocks!/h3\n; Browsing to localhost/perl/rocks.html gives: Content-type: text/html mod_perl 2.0 rocks! (with the second line emboldened to h3). The view-source shows: Content-type: text/html h3mod_perl 2.0 rocks!/h3 Question: why isn't my 'Content-type' line output as a HTTP header? [...] I've fixed it. Please try with the latest cvs. __ Stas BekmanJAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide --- http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com Need you ask? Just recompiled/reinstalled mp2-cvs - rocks.html *and* rocks.pl both work! Aloha = Beau;
Re: [mp2] CGI redirects incorrectly handled?
Mark James wrote: Stas Bekman wrote: Can you send a short script (removing all the irrelevant bits) that we can reproduce the problem with? Made a script that generated the same POST request and same redirect as the problem code. The problem was not reproduced! The only difference I can see between working POSTs (both those in my package code and the one in the test script) and the failing one (a particular one in my package) is in the distribution of the data across the TCP packets that carry the POST. [snippet the packet dumps] Could mod_perl, with its persistent buffer, be tripping up on this? I'm trying now to trace the data through the mp2 code. -- Mark mod_perl 2.0 is buffering only the content (response body, not the headers) (assuming that you aren't installing any output filters), so it's one of the apache core output filters, that decides how to split the data. a normal output filter list ends with: byterange(0x8841110): ctx=0x0, r=0x88402d0, c=0x883a390 content_length(0x8841128): ctx=0x0, r=0x88402d0, c=0x883a390 http_header(0x8841140): ctx=0x0, r=0x88402d0, c=0x883a390 core(0x883a760): ctx=0x883a738, r=0x0, c=0x883a390 you can dump this, using the dump_filters macro, which you can load by running gdb source /path/to/httpd-2.0/.gdbinit the 'core' filter is in httpd-2.0/src/core.c: core_output_filter(...) __ Stas BekmanJAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide --- http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com
Re: Trouble with sysread in modperl
Liu, Hui (GXS) wrote:
Stas,
Yes we tried reading STDIN and it works:
while ($readData = STDIN) {
$buffer.= $readData;
$bufferLength = length($buffer);
..
}
But it's not working with sysread or read, here are the results
for read, the last line is what we expected but it's null:
while ($bytesRead = read(STDIN, $buffer, 4096)) {
html(bytesRead=[$bytesRead] data=[$buffer]) if ($debug);
}
Here's the results from the above code :
I suggest that you look at the implementation of the function read_multipart()
in CGI.pm, which does exactly what you want. (I see no reason why not using
CGI.pm in first place if you can.)
And please, refrain from posting emails with huge chunks of useless
information, in the future. Truncate to keep only the information that may
help to understand and reproduce the problem.
__
Stas BekmanJAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide --- http://perl.apache.org
mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
[mp2] changing http:// to https: in TransHandler
Hi -
I'm not much of a mod_perl scripter (yet), but having been
totally defeated my mod_rewrite, I am trying to use mod_perl
to push clients into using https when accessing a particular
server (I am using named-based virtual hosting).
I want to do something like this (the real one will be
more complicated - but this is a baby test):
-in httpd.conf-
PerlTransHandler +MyApache::ForceSecure
-handler-
package MyApache::ForceSecure;
use strict;
use warnings;
use Apache::RequestRec ();
use Apache::Const -compile = qw(DECLINED);
sub handler
{
my $r = shift;
my $url = $r-url;
if ($url =~ m{^http://bcbk}i) {
$url =~ s/^http:/https:/i;
$r-url ($url);
}
return Apache::DECLINED;
}
1;
Which is great, but there is *no* $r-url. I know there is a $r-uri, but
how can I get to the whole ball of wax: from http://...? I can't find
it in the docs.
Aloha = Beau;
Re: Optimising cache performance
What implications does this have on the size of the cache that can be created with IPC::MM I believe that documentation is telling you that each OS governs the amount of shared memory you can have in different ways. Linux, for example, has a variable called shmmax, accessible as /proc/sys/kernel/shmmax, which controls how much shared memory you are allowed to allocate. I think Solaris' setting lives in /etc/system somewhere. Cory 'G' Watson http://gcdb.spleck.net
Re: [mp2] changing http:// to https: in TransHandler
On Sat, 8 Mar 2003 [EMAIL PROTECTED] wrote:
Hi -
I'm not much of a mod_perl scripter (yet), but having been
totally defeated my mod_rewrite, I am trying to use mod_perl
to push clients into using https when accessing a particular
server (I am using named-based virtual hosting).
I want to do something like this (the real one will be
more complicated - but this is a baby test):
-in httpd.conf-
PerlTransHandler +MyApache::ForceSecure
-handler-
package MyApache::ForceSecure;
use strict;
use warnings;
use Apache::RequestRec ();
use Apache::Const -compile = qw(DECLINED);
sub handler
{
my $r = shift;
my $url = $r-url;
if ($url =~ m{^http://bcbk}i) {
$url =~ s/^http:/https:/i;
$r-url ($url);
}
return Apache::DECLINED;
}
1;
Which is great, but there is *no* $r-url. I know there is a $r-uri, but
how can I get to the whole ball of wax: from http://...? I can't find
it in the docs.
Aloha = Beau;
Beau:
I _just_ went through this on my system. You would probably want to use
the following to change the URI as you wish:
my $uri = APR::URI-parse($r-pool, $r-construct_url);
$uri-scheme('https');
my $new_uri = $uri-unparse;
However, the overall strategy is probably not what you want, due to the
way SSL works. When a browser requests a secure connection, the SSL
connection (to the secure port) is established _before_ even the HTTP
connection. Thus it is impossible to change the scheme (http vs https)
once you have arrived at your server. The only way to do this with a Perl
handler is to generate a 302 external redirect.
mod_rewrite can be complicated, sure, but I do think it's the way to
go in this situation. You need:
- two sub-domains in DNS, let's say www.my_domain.com and secure.my_domain.com
- a sub-directory /secure in your webdocs root (or something else able to matched with
a regex)
- the following in your httpd.conf:
Listen 80
Listen 443
NameVirtualHost 12.34.56.789:80
NameVirtualHost 12.34.56.789:443
VirtualHost 12.34.56.789:80
ServerName www.my_domain.com
RewriteEngine on
RewriteCond %{REQUEST_URI} /secure/
RewriteRule ^/(.*)$ https://secure.my_domain.com/$1 [R,L]
/VirtualHost
VirtualHost 12.34.56.789:443
ServerName secure.my_domain.com
RewriteEngine on
RewriteCond %{REQUEST_URI} !/secure
RewriteRule ^/(.*)$ http://www.my_domain.com/$1 [R,L]
/VirtualHost
This allows you to have relative links on all your pages. All links on
www.my_domain.com will point to http://www. on port 80, and all links on
secure.my_domain.com will point to https://secure. on port 443. The server
will simply rewrite and redirect all links that do not match either
/secure/ or !/secure.
Hope this helps,
- nick
PS If you have more than one domain needing to use https, you can put it
on an arbitrary port so long as you configure the server (not apache) to
listen on it, and then hard-code the port number in the mod_rewrite rule.
--
Nick Tonkin {|8^)
Re: [mp2] changing http:// to https: in TransHandler
On 8 Mar 2003 at 6:45, Nick Tonkin wrote:
On Sat, 8 Mar 2003 [EMAIL PROTECTED] wrote:
Hi -
I'm not much of a mod_perl scripter (yet), but having been
totally defeated my mod_rewrite, I am trying to use mod_perl
to push clients into using https when accessing a particular
server (I am using named-based virtual hosting).
[...]
I _just_ went through this on my system. You would probably want to use
the following to change the URI as you wish:
my $uri = APR::URI-parse($r-pool, $r-construct_url);
$uri-scheme('https');
my $new_uri = $uri-unparse;
However, the overall strategy is probably not what you want, due to the
way SSL works. When a browser requests a secure connection, the SSL
connection (to the secure port) is established _before_ even the HTTP
connection. Thus it is impossible to change the scheme (http vs https)
once you have arrived at your server. The only way to do this with a Perl
handler is to generate a 302 external redirect.
mod_rewrite can be complicated, sure, but I do think it's the way to
go in this situation. You need:
[...]
Nick Tonkin {|8^)
Thank you Nick for your detailed and informative reply! Back to mod_rewrite ;)
I'll see if I can get thru the virtual host/mod_rewrite maze...and
let you know.
Thanks and Aloha = Beau;
Re: how to take advantage of mod_perl and analize effectiveness ofefforts?
On Fri, 2003-03-07 at 18:21, Charlie Smith wrote: What is being cached by the mod_perl? You should definitely read the mod_perl documentation that another poster pointed you to. However, let me address your core question about what is being cached with mod_perl. Technically, mod_perl doesn't cache any differently from Perl itself. When Perl code gets compiled to bytecode it stays in memory until the Perl interpreter exits. The key difference between mod_perl and Perl CGI is that mod_perl keeps the interpeter alive between requests and thus keeps the compiled bytecode in memory as well. This is what people mean by caching code. Some frameworks that you can use with mod_perl, like Mason or Embperl, cache other things as well. This is explained in their documentation. There are ways of configuring mod_perl to automatically pick up changes in your code. This is all described in the mod_perl documentation. - Perrin
Re: DECLINE, ERRORs in the middle of hanlers stack
Ruslan U. Zakirov wrote: On Sat, 8 Mar 2003, Stas Bekman wrote: Ruslan U. Zakirov wrote: Hello All! Stacked handlers is a very useful technology, but as I think incomplete. I need some suggestions. My project split content handler in the few parts. And each handler send part of the requested page to user, but sometimes I must stop proccessing and return DECLINE, redirect user or return some ERROR. Error appear in the middle of the page. I want Apache to buffer content and send it after last handler in chain return OK. Is it possible? Sure, you can buffer the data using $r-notes or $r-pnotes and have the last handler send it out. But instead of stuffing the content there, you are probably better off (more efficient?) to create a buffering class, instantiate an instance, and store it in $r-pnotes between invocations. Hello Stas and other! Thanks for idea, but it force me to change all scripts. Another alternative is to try using Apache::Filter for the buffering and $r-notes/pnotes for flow control (so you can inform Apache::Filter what to do with the buffered data). mod_perl 2.0 filters are probably more flexible, but I see no reason why mp1's Apache::Filter won't do the job. __ Stas BekmanJAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide --- http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com
RE: Trouble with sysread in modperl
Title: RE: Trouble with sysread in modperl
Stas,
Does this mean read or sysread not work properly in mod_perl and we should stay away from the two functions if we write perl code to work with Apache server?
Regards,
Hui
-Original Message-
From: Stas Bekman [mailto:[EMAIL PROTECTED]]
Sent: Saturday, March 08, 2003 7:09 AM
To: Liu, Hui (GXS)
Cc: [EMAIL PROTECTED]
Subject: Re: Trouble with sysread in modperl
Liu, Hui (GXS) wrote:
Stas,
Yes we tried reading STDIN and it works:
while ($readData = STDIN) {
$buffer .= $readData;
$bufferLength = length($buffer);
..
}
But it's not working with sysread or read, here are the results
for read, the last line is what we expected but it's null:
while ($bytesRead = read(STDIN, $buffer, 4096)) {
html(bytesRead=[$bytesRead] data="" if ($debug);
}
Here's the results from the above code :
I suggest that you look at the implementation of the function read_multipart()
in CGI.pm, which does exactly what you want. (I see no reason why not using
CGI.pm in first place if you can.)
And please, refrain from posting emails with huge chunks of useless
information, in the future. Truncate to keep only the information that may
help to understand and reproduce the problem.
__
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide --- http://perl.apache.org
mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
Re: [mp2] changing http:// to https: in TransHandler
sorry if OT..
Hi Nick,
please tell me I'm wrong (I'll be a happy camper), but I thought that you
couldn't use name virtual server for SSL.
Name server requires HTTP/1.1 which supplies a Host header so the server can
tell which virtual server you want. With SSL this header is encrypted so
apache can't read it to know which virtual server it's for.
Or does it work this way by defaulting to the first virtual server listening
on port 443?
Or is Apache2 doing something funky to make this work?
..again, I really would like to be wrong about this. I host from home on ADSL
and thought I'd have to pay for more IP's if I wanted to secure a section of
my site.
J
Nick Tonkin wrote:
On Sat, 8 Mar 2003 [EMAIL PROTECTED] wrote:
Hi -
I'm not much of a mod_perl scripter (yet), but having been
totally defeated my mod_rewrite, I am trying to use mod_perl
to push clients into using https when accessing a particular
server (I am using named-based virtual hosting).
I want to do something like this (the real one will be
more complicated - but this is a baby test):
-in httpd.conf-
PerlTransHandler +MyApache::ForceSecure
-handler-
package MyApache::ForceSecure;
use strict;
use warnings;
use Apache::RequestRec ();
use Apache::Const -compile = qw(DECLINED);
sub handler
{
my $r = shift;
my $url = $r-url;
if ($url =~ m{^http://bcbk}i) {
$url =~ s/^http:/https:/i;
$r-url ($url);
}
return Apache::DECLINED;
}
1;
Which is great, but there is *no* $r-url. I know there is a $r-uri, but
how can I get to the whole ball of wax: from http://...? I can't find
it in the docs.
Aloha = Beau;
Beau:
I _just_ went through this on my system. You would probably want to use
the following to change the URI as you wish:
my $uri = APR::URI-parse($r-pool, $r-construct_url);
$uri-scheme('https');
my $new_uri = $uri-unparse;
However, the overall strategy is probably not what you want, due to the
way SSL works. When a browser requests a secure connection, the SSL
connection (to the secure port) is established _before_ even the HTTP
connection. Thus it is impossible to change the scheme (http vs https)
once you have arrived at your server. The only way to do this with a Perl
handler is to generate a 302 external redirect.
mod_rewrite can be complicated, sure, but I do think it's the way to
go in this situation. You need:
- two sub-domains in DNS, let's say www.my_domain.com and secure.my_domain.com
- a sub-directory /secure in your webdocs root (or something else able to matched with
a regex)
- the following in your httpd.conf:
Listen 80
Listen 443
NameVirtualHost 12.34.56.789:80
NameVirtualHost 12.34.56.789:443
VirtualHost 12.34.56.789:80
ServerName www.my_domain.com
RewriteEngine on
RewriteCond %{REQUEST_URI} /secure/
RewriteRule ^/(.*)$ https://secure.my_domain.com/$1 [R,L]
/VirtualHost
VirtualHost 12.34.56.789:443
ServerName secure.my_domain.com
RewriteEngine on
RewriteCond %{REQUEST_URI} !/secure
RewriteRule ^/(.*)$ http://www.my_domain.com/$1 [R,L]
/VirtualHost
This allows you to have relative links on all your pages. All links on
www.my_domain.com will point to http://www. on port 80, and all links on
secure.my_domain.com will point to https://secure. on port 443. The server
will simply rewrite and redirect all links that do not match either
/secure/ or !/secure.
Hope this helps,
- nick
PS If you have more than one domain needing to use https, you can put it
on an arbitrary port so long as you configure the server (not apache) to
listen on it, and then hard-code the port number in the mod_rewrite rule.
Re: Optimising cache performance
Clinton Gormley wrote: For now it's not a distributed system, and I have been using Cache::FileCache. But that still means freezing and thawing objects - which I'm trying to minimise. Other things (IPC::MM, MLDBM::Sync, Cache::Mmap, BerkeleyDB) are significantly faster than Cache::FileCache. If you have tons of free memory, then go ahead and cache things in memory. My feeling is that the very small amount of time that the fastest of these systems use to freeze and thaw is totally made up for in the huge memory savings which allows you to run more server processes. When you say that Cache::Mmap is only limited by the size of your disk, is that because the file in memory gets written to disk as part of VM? ( I don't see any other mention of files in the docs.) Which presumably means resizing your VM to make space for the cache? That's right, it uses your system's mmap() call. I've never needed to adjust the amount of VM I have because of memory-mapping a file, but I suppose it could happen. This would be a good question for the author of the module, or an expert on your system's mmap() implementation. I see the author of IPC::MM has an e-toys address - was this something you used at e-toys? It was used at one point, although not in the version of the system that I wrote about. He originally wrote it as a wrapper around the mm library, and I asked if he could put in a shared hash just for fun. It turned out be very fast, largely because the sharing and the hash (or btree) is implemented in C. The Perl part is just an interface to it. I know very little about shared memory segments, but is MM used to share small data objects, rather than to keep large caches in shared memory? It's a shared hash. You can put whatever you want into it. Apache uses mm to share data between processes. Ralph Engelschall writes in the MM documentation : The maximum size of a continuous shared memory segment one can allocate depends on the underlaying platform. This cannot be changed, of course. But currently the high-level malloc(3)-style API just uses a single shared memory segment as the underlaying data structure for an MM object which means that the maximum amount of memory an MM object represents also depends on the platform. What implications does this have on the size of the cache that can be created with IPC::MM It varies by platform, but I believe that on Linux it means each individual hash is limited to 64MB. So maybe I spoke too soon about having unlimited storage, but you should be able to have as many hashes as you want. If you're seriously concerned about storage limits like these, you could use one of the other options like MLDBM::Sync or BerkeleyDB which use disk-storage. - Perrin
Re: [mp2] changing http:// to https: in TransHandler
On 9 Mar 2003 at 10:53, Jason Galea wrote:
sorry if OT..
Hi Nick,
please tell me I'm wrong (I'll be a happy camper), but I thought that you
couldn't use name virtual server for SSL.
Name server requires HTTP/1.1 which supplies a Host header so the server can
tell which virtual server you want. With SSL this header is encrypted so
apache can't read it to know which virtual server it's for.
Or does it work this way by defaulting to the first virtual server listening
on port 443?
Or is Apache2 doing something funky to make this work?
..again, I really would like to be wrong about this. I host from home on ADSL
and thought I'd have to pay for more IP's if I wanted to secure a section of
my site.
J
Nick Tonkin wrote:
[...]
Beau:
[...]
mod_rewrite can be complicated, sure, but I do think it's the way to
go in this situation. You need:
- two sub-domains in DNS, let's say www.my_domain.com and secure.my_domain.com
- a sub-directory /secure in your webdocs root (or something else able to matched
with a regex)
- the following in your httpd.conf:
Listen 80
Listen 443
NameVirtualHost 12.34.56.789:80
NameVirtualHost 12.34.56.789:443
VirtualHost 12.34.56.789:80
ServerName www.my_domain.com
RewriteEngine on
RewriteCond %{REQUEST_URI} /secure/
RewriteRule ^/(.*)$ https://secure.my_domain.com/$1 [R,L]
/VirtualHost
VirtualHost 12.34.56.789:443
ServerName secure.my_domain.com
RewriteEngine on
RewriteCond %{REQUEST_URI} !/secure
RewriteRule ^/(.*)$ http://www.my_domain.com/$1 [R,L]
/VirtualHost
This allows you to have relative links on all your pages. All links on
www.my_domain.com will point to http://www. on port 80, and all links on
secure.my_domain.com will point to https://secure. on port 443. The server
will simply rewrite and redirect all links that do not match either
/secure/ or !/secure.
Hope this helps,
- nick
PS If you have more than one domain needing to use https, you can put it
on an arbitrary port so long as you configure the server (not apache) to
listen on it, and then hard-code the port number in the mod_rewrite rule.
I'm not Nick and you're wrong! :)
Just follow Nick's cookbook above, and it will work.
I put all of my non-global SSL directives within the
secure vhost block.
You may have to tweak it your your particular needs,
but, hey, that's fun anyway...
Aloha = Beau;
Re: [mp2] changing http:// to https: in TransHandler
On Sun, 9 Mar 2003, Jason Galea wrote:
sorry if OT..
Yes, it's OT. Please take SSL questions to an ssl-related list. Or, since
the previous post contained cut-n-paste instructions, you could have tried
it! :)
please tell me I'm wrong (I'll be a happy camper), but I thought that you
couldn't use name virtual server for SSL.
The basic answer to your question is that you only need unique IP-port
combinations to run multiple SSL virtual hosts using NameVirtualHost.
However, requests to any SSL virtual host other than the one running on
port 443 (the standard https port) will have to specify the port in the
request.
I suggest spending some time with the docs for mod_ssl, if that's what
you're using.
- nick
--
Nick Tonkin {|8^)
Re: Trouble with sysread in modperl
Liu, Hui (GXS) wrote: Stas, Does this mean read or sysread not work properly in mod_perl and we should stay away from the two functions if we write perl code to work with Apache server? I haven't said that. I doubt so. It could be a bug in your code. I've suggested: I suggest that you look at the implementation of the function read_multipart() in CGI.pm, which does exactly what you want. (I see no reason why not using CGI.pm in first place if you can.) Have you tried that? Also I'm assuming that you are using mod_perl 1.x, right? __ Stas BekmanJAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide --- http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com
