Re: Apache::SSI pain
The thing that works best for me is to use an Apache::SSI object myself
rather than using any kind of chaining.
my $ssi = Apache::SSI-new($text, $r);
print $q-header; # send the HTTP header
$ssi-output();
Or you can capture the output instead of just sending it, like this:
my $ssi= Apache::SSI-new($text, $r);
my $result = $ssi-get_output();
I actually do this with a customized subclass that implements some
site-specific SSI directives. I'm calling it from within a custom
handler, and not inside any kind of Registry or PerlRun script. Works
perfectly for me. YMMV.
- Perrin
Christopher Hicks wrote:
I've used the lines below and the various other provided incantations for
using Apache::SSI or Apache::SSIChain.
PerlModule Apache::SSIChain
PerlModule Apache::Filter
PerlModule Apache::SSI
PerlModule Apache::OutputChain
Alias /ssiperl/ /www/perl/ssibin/
Location /ssiperl
SetHandler perl-script
# PerlSendHeader On
PerlSetVar Filter On
# PerlHandler Apache::OutputChain Apache::SSIChain Apache::Registry
# PerlHandler Apache::SSIChain Apache::Registry
PerlHandler Apache::Registry Apache::SSI
Options +ExecCGI
/Location
The SSI's get included, but they're all at the top of the file. Both of
the PerlHandler lines commented out above have SSIChain as the last thing
which is what the docs say to do. But the includes still get included
before the rest of the file.
I've scoured the modperl archives. I've read through the source and
documentation to Apache::SSI*. Does anyone have any ideas?
My workaround:
sub processSSI {
my ($text,$r) = @_;
my @lines = split(/\n/,$text);
my ($line,@el,$command,$key,$val,%param);
foreach $line (@lines) {
if ($line =~ /!--#(.*)--/) {
@el = split(/\s+/,$1);
$command = lc(shift(@el));
foreach (@el) {
($key,$val) = split(/=/);
$key = lc($key); # lower case key
# strip double quotes if they're there
# and balanced
if (substr($val,0,1) eq '"') {
if (substr($val,-1,1) eq '"') {
chop($val);
substr($val,0,1) = '';
}
}
$param{$key}=$val;
}
if ($command eq "include") {
if (defined( $param{'virtual'} )) {
Apache::Include-virtual($param{'virtual'});
}
}
} else {
print $line, "\n";
}
}
}
This handles my needs. It is a disadvantage to be stuck with putting
anything that isn't an "include virtual" into a file which is then called
via an "include virtual". But at least it works.
--
/chris
"The only thing more frightening than a programmer with a screwdriver
or a hardware enginner with a program is a user with wire cutters and
the root password."- Elizabeth Zwicky
Re: Apache::DBI MySQL
Viren Jain wrote: I included the command "PerlModule Apache::DBI" in my mod_perl Apache configuration files. Yet, over time there builds up more connection in mysql than apache processes (only Apache/CGI should be accessing MySQL) and most processes seem to have very high "Time"s under the "sleep" stat when I do a 'processlist' in mysql. Please advise. -- Viren If ANYTHING is different in your connect string, Apache::DBI will keep a separate connection open. You probably have slight variations in your connect strings. - Perrin
Re: More on web application performance with DBI
On Thu, 14 Oct 1999, Jeffrey Baker wrote: Zero optimization: 41.67 requests/second Stage 1 (persistent connections): 140.17 requests/second Stage 2 (bound parameters): 139.20 requests/second Stage 3 (persistent statement handles): 251.13 requests/second I know you said you don't like it because it has extra overhead, but would you mind trying stage 3 with prepare_cached rather than your custom solution with globals? For some applications with lots of SQL statements, the prepare_cached appraoch is just much more manageable. It is interesting that the Stage 2 optimization didn't gain anything over Stage 1. I think Oracle 8 is doing some magic by parsing your SQL and matching it up to previous statements, whether you use bind variables or not. It may matter more on other databases. - Perrin
Re: RegistryNG docs?
Ken Williams wrote: The only thing keeping RegistryNG from being the main version of Registry (which it will be in the future) is that it hasn't been tested enough by people like you. So go for it! It's probably going to be a much cleaner way to proceed than mucking around in Registry. You know, I tried RegistryBB (which is a subclass of RegistryNG). I was seeing the call to Apache::constants::OPT_EXECCGI at the top of my DProf output, and I wanted to get rid of it. Unfortunately, RegistryBB was actually slower for my app than the normal Registry. Maybe that's not typical, but for me the overhead of the method call seemed to erase any savings from not stat-ing files, etc. - Perrin
Re: Summary of DB_File locking wrappers
David Harris wrote: There are three locking wrappers for DB_File in CPAN right now. Each one implements locking differently and has different goals in mind. It is therefore worth knowing the difference, so that you can pick the right one for your application. Good summary. Thanks! Note that you could also use BerkeleyDB (interface to version 2 3 of Berkeley DB library), which manages its own locks at a page level using shared memory. I haven't used this under load, so I can't vouch for it yet, but it looks like a great solution for sites that run on one machine. - Perrin
Re: Summary of DB_File locking wrappers
On Tue, 11 Jan 2000, Stas Bekman wrote: There are three locking wrappers for DB_File in CPAN right now. Each one implements locking differently and has different goals in mind. It is therefore worth knowing the difference, so that you can pick the right one for your application. Good summary. Thanks! Note that you could also use BerkeleyDB (interface to version 2 3 of Berkeley DB library), which manages its own locks at a page level using shared memory. I haven't used this under load, so I can't vouch for it yet, but it looks like a great solution for sites that run on one machine. Can you please put more light on this issue? Sounds very interesting. I didn't have a chance to use BerkeleyDB, since it claims to be a driver in alpha stage. So is it stable enough, to be used on production machine? I haven't used it enough to make that call. If you want to try out the locking, keep in mind that you have to do some setup work beyond a simple tie in order to initialize the environment. My information about its capabilities is mostly from the Sleepycat Software site. Check out http://www.sleepycat.com/featdetail.html#concurrent_write for more. - Perrin
Re: problems with module at root of web site
Sean Chittenden wrote: Mind if I ask a nit-pick of a performance question? Currently speed and performance are of upmost importance (I'm currently involved in a mod_perl vs JServ development race). If you're on Linux, I can tell you right now that mod_perl is significantly faster than the fastest servlet runner (http://www.caucho.com/) with the fastest JVM (IBM 1.1.8 final release). If you're on Solaris, all bets are off. I suspect you should be able to clobber JServ in the performance department on any platform though. - Perrin
Re: ANNOUNCE: HTML::Mason 0.8
On Sun, 23 Jan 2000, Jonathan Swartz wrote: - New in-memory code cache keeps track of component usage, and discards the most infrequently used components as needed. You can specify the cache size with interp-max_code_cache_size. This sounds cool, but does it work, i.e. when you undef a sub reference, does the memory really get released and reused? - Perrin
Re: mod_perl flashback
now, i have the exact same problem: i need my SSI to filter thru everithing: HTML, CGIs, PHP, etc. You get HTML filtering already. For CGIs, why not write a minimal PerlHandler to emulate CGI (i.e. set up the environment and execute the CGI script) and then run the output through SSI? For parsing the output of actual apache modules like PHP, I have no idea, but PHP does have a CGI mode. - Perrin
Re: oracle : The lowdown
Perrin Harkins wrote: Greg Stark wrote: For example, it makes it very hard to mix any kind of long running query with OLTP transactions against the same data, since rollback data accumulates very quickly. I would give some appendage for a while to tell Oracle to just use the most recent data for a long running query without attempting to rollback to a consistent view. I believe setting the isolation level for dirty reads will allow you to do exactly that. Oh, silly me. Oracle doesn't appear to offer dirty reads. The lowest level of isolation is "read committed" which reads all data that was committed at the time the query began, but doesn't preserve that state for future queries. So, if you have lots of uncommitted data or you commit lots of data to the table being queried while the query is running you could make your rollback segment pretty big. But, if you can afford Oracle, you can afford RAM. - Perrin
Re: ApacheDBI question
On Fri, 28 Jan 2000, Deepak Gupta wrote: How does connection pooling determine how many connections to keep open? The reason I ask is that I am afraid my non-modperl scripts are getting rejected by the db server b/c all (or most) connections are being dedicated to Apache activity. Please RTFM. perldoc Apache::DBI. - Perrin
Re: splitting mod_perl and sql over machines
On Fri, 28 Jan 2000, Marko van der Puil wrote: There has been an discussion in the Mod_Perl mailing list about whether you would profit from splitting your Mod_Perl enabled Apache server and a SQL database like MySQL over multiple machines. To give this discussion some technical and scientific background I've run some benchmarks. They're available in HTML on http://cztb.zeelandnet.nl/rzweb/benchmarks/splitservers.html While I agree with your conclusion (splitting the database onto another machine is a given in a clustered environment like the one I work with), I think your benchmark is kind of misleading. You test performance with everything on one machine and then you test it with things split between two machines. To be fair, both tests should use both machines. - Perrin
Re: Novel technique for dynamic web page generation
On Fri, 28 Jan 2000, Paul J. Lucas wrote:
but it seems like it does tie your program closely to the structure of the
documents.
It does somewhat, but much less so than existing techniques:
1. Conventional CGI (a print-statement-laden Perl script): this
tightly intertwines code and markup.
2. Embedded Perl (e.g., ePerl): variables are placed at
specific points in the markup.
3. Non-standard tags: placed at specific points in the markup.
(Another downside: DreamWeaver doesn't understand them.)
Now that I've seen your example, it seems to me that you are doing almost
exactly the same as #3. The only difference is that you're using HTML
extensions ("CLASS=foo") that are legal in authoring tools. Otherwise,
this is really the same effect as using HTML::Template or Template
Toolkit.
Am I correct in thinking that if you want to put a piece of text pulled from
a database into a page you have to know exactly where it should end up in the
HTML parse tree?
Yes; but you control where that is by editing the HTML file.
And you can reedit it and move it around again and again; and
you never have to touch the underlying Perl code. See above.
This is different from XMLC, which requires the program using it to
specify which node to replace the contents of. I think your approach is
better.
- Perrin
Re: ANNOUNCE: Updated Hello World Web Application Benchmarks
Joshua Chamas wrote: There is no way that people are going to benchmark 10+ different environments themselves, so this merely offers a quick fix to get people going with their own comparisons. I agree that having the code snippets for running hello world on different tools collected in one place is handy. Do you have any idea how much time it takes to do these? Yes, I've done quite a few of them. I never said they were easy. In order to improve the benchmarks, like the Resin Velocigen ones that you cited where we have a very small sample, we simply need more numbers from more people. I think we would need more numbers from the exact same people, on the same machines, with the same configuration, the same client, the same network, the same Linux kernel... In other words, controlled conditions. Also, any disclaimer modifications might be good if you feel there can be more work done there. Ideally, I would get rid of every page except the one which lists the tests grouped by OS/machine. Then I would put a big statement at the top saying that comparisons across different people's tests are meaningless. - Perrin
Re: ANNOUNCE: Updated Hello World Web Application Benchmarks
I think too that the OS/machine results at http://www.chamas.com/bench/hello_bycode.html could be more accurate in comparing results if the results are also grouped by tester, network connection type, and testing client so each grouping would well reflect the relative speed differences web applications on the same platform. Agreed. I would argue that we should keep the code type grouping listed at http://www.chamas.com/bench/hello_bycode.html because it gives a good feel for how some operating systems web servers are faster than others, i.e., Solaris slower than Linux, WinNT good for static HTML, Apache::ASP faster than IIS/ASP PerlScript, etc. See, I don't think you can even make statements like that based on these benchmarks. Where is the test on Solaris x86 and Linux done by the same person under the same conditions? I don't see one. Where is the test of NT and Linux on the same machine by the same person? Even the Apache::ASP vs PerlScript comparisons you did seem to be using different clients, netowork setups, and versions of NT. I'm not criticizing you for not being able to get lab-quality results, but I think we have to be careful what conclusions we draw from these. Finally, I would very much like to keep the fastest benchmark page as the first page, disclaiming it to death if necessary, the reason being that I would like to encourage future submissions, with new faster hardware OS configurations, and the best way to do that is to have something of a benchmark competition happening on the first page of the results. I can understand that; I just don't want mod_perl users to get a reputation as the Mindcraft of web application benchmarks. It seems that HTTP 1.1 submissions represent a small subset of skewed results, should these be dropped or presented separately? I'd say they're as meaningful as any of the others if you consider them independently of the other contributions. I also need to clarify some results, or back them up somehow. What should I do with results that seem skewed in general? Not post them until there is secondary confirmation ? Your call. Again, to my mind each person's contribution can only be viewed in its own private context, so one is no more skewed than any other. - Perrin
Re: ANNOUNCE: Updated Hello World Web Application Benchmarks
On Sun, 30 Jan 2000, Perrin Harkins wrote: I can understand that; I just don't want mod_perl users to get a reputation as the Mindcraft of web application benchmarks. I'm not sure I see how that can happen when we quite clearly state that php4 is faster than mod_perl. Only one person bothered to do both php and a PerlHandler, and in his test mod_perl came out a little bit ahead. On the "fastest" page, PerlHandler gas a much higher score than php. This is the kind of confusion I'm talking about. - Perrin
Re: IPC::Cache version 0.01 available
Hi, A week or two ago, in the squid performance thread, I mentioned that I was looking for ways to eliminate squid from our production servers. I noted that we are using squid to save an expensive trip to the database to retrieve mostly static files. At that time I said that I planned to write a simple cache using shared memory that could be used with mod_perl to keep data around on the first tier. This weekend I finally had an opportunity to do just that. IPC::Cache is rather simple, considering that all the hard work is being done in the IPC::Shareable the Storable modules. I just added some basic namespace semantics and the ability to have data expire after a period of time. I'm planning on submitting it to CPAN, but I think it makes sense to make it available on this list before I do. I'd appreciate it if anyone who tries it out could send me feedback about whether it seems to do the trick. For now, you can get it at: http://unto.net/archives/IPC-Cache-0.01.tar.gz Apologies if that site is a little slow -- I'll move it over to our real servers and to CPAN in the near future. Thanks! -DeWitt Two suggestions: - Make it compatible with Apache::Session. - Try using IPC::MM rather than IPC::Shareable. It may be faster, if you don't need to store complex datatypes.
Re: Using network appliance Filer with modperl
On Mon, 31 Jan 2000, siberian wrote: My question is : Has anyone experienced any 'gotchas' in putting perl code that modperl handlers use on a Network Attached file server like a network appliance box ( www.netapp.com )? Just make sure you have time synchronized between your servers and the NetApp. Otherwise, things that use file mod times (like Registry) will have strange behavior. - Perrin
Re: Caucho faster than mod_perl?
Sean Chittenden wrote:
http://www.caucho.com/articles/benchmark.html
Supposedly, according to its benchmarks, it's faster than mod_perl...
impressive to say the least. Any chance someone has any experience
with this or would like to benchmark this technology? External validation
would be pretty useful.
It is not faster than mod_perl. We tested it and found that the Resin
author used code that didn't take good advantage of mod_perl, i.e.
Apache::Registry scripts rather than PerlHandler modules. We sent our
results and code to Caucho, and received a cordial reply, but they have
not updated their page. The following is an excerpt from the message one
of our engineers sent to them:
I've recently been testing Resin 1.1b4's performance compared with
mod_perl. The "benchmark.jar" from your web site included perl scripts
that really didn't take full advantage of the capabilities that mod_perl
offers. So, I wrote three perl modules that attempt to be similar to
examples included in the Resin 1.1b4 distribution and better reflect
what mod_perl can do.
Attached is a gzipped tar file with these perl modules, and you're free
to do whatever you want with them.
Overall, I've found Resin to be a very good product, yet ran into a few
problems during testing. These may be caused by the IBM 1.1.8 JDK rather
than the Resin package itself. For example when running Resin as a
stand-alone httpd server, then using "httperf" to benchmark it, the
entire JVM would freeze up when the --close-with-reset option was given
to httperf.
The JVM used in testing is IBM 1.1.8, on Red Hat Linux 6.0, 2x500MHz
Pentium III, 512MB RAM, Oracle 8.0.5 client libraries (JDBC). Apache
1.3.9 with DSO support was used for "Resin+Apache", with 60 max clients.
A recent version of mod_perl and Apache (without DSO) was used for the
"mod_perl+Apache" tests, also with 60 max clients. Finally, "Resin
httpd" is Resin running its own web server, with max threads set to 60.
These tests used httperf with persistent connections, making either
1000 or 100 requests per connection, depending on the test. The results
on "hello" are astounding:
Test: "hello" -- displays "Hello, World"
Size: 450 bytes
httperf: 40 concurrent connections, 1000 requests/connection
Results:
mod_perl+Apache: 1600 req/sec
Resin+Apache: 500 req/sec
Resin httpd: 4600 req/sec
Test: "bighello" -- lots of HTML + "Hello World"
Size: 23888 bytes
httperf: 40 concurrent connections, 100 requests/connection
Results:
mod_perl+Apache: 480 req/sec
Resin+Apache: 300 req/sec
Resin httpd: 280 req/sec
Test: "database" -- query: "select NAME from DIVISION", returns 11
rows
Size: 460 bytes
httperf: 40 concurrent connections, 100 requests/connection
Results:
mod_perl+Apache: 570 req/sec
Resin+Apache: 300 req/sec
Resin httpd: 450 req/sec
So, although "Resin httpd" performs very well with many small transfers
("hello"), the performance relative to mod_perl appears to degrade with
larger transfers ("bighello"). The database performance improved nicely
from where it was under Resin+Apache, but mod_perl still has the edge
there.
I also tested the "file.jsp", which I renamed "fortune" in the perl
tests. With http_load, the results again show mod_perl ahead:
Resin:
1584 fetches, 10 max parallel, 489610 bytes, in 10 seconds
309.097 mean bytes/connection
158.4 fetches/sec, 48961 bytes/sec
mod_perl:
6190 fetches, 10 max parallel, 1.98814e+06 bytes, in 10 seconds
321.186 mean bytes/connection
619 fetches/sec, 198814 bytes/sec
I'm not trying to belittle Resin in any way -- in fact I'm impressed by
both its design and performance. In learning about Resin during these
tests, I found that JSP is in many ways easier to use than mod_perl. The
smart caching that Resin does (with compiling .java - .class, etc.) is
effective, and the XML configuration is a joy to deal with compared to
Apache's httpd.conf.
So, it's nice, but not as fast as mod_perl on Linux. If anyone wants to
try this on another OS, I can supply a tar of the files we used for
testing.
- Perrin
Re: Caucho faster than mod_perl?
Sean Chittenden wrote:
http://www.caucho.com/articles/benchmark.html
Supposedly, according to its benchmarks, it's faster than mod_perl...
impressive to say the least. Any chance someone has any experience with this
or would like to benchmark this technology? External validation would be
pretty useful. --SC
It is not faster than mod_perl. We tested it and found that the Resin
author used code that didn't take good advantage of mod_perl, i.e.
Apache::Registry scripts rather than PerlHandler modules. We sent our
results and code to Caucho, and received a cordial reply, but they have
not updated their page. The following is an excerpt from the message
one of our engineers sent to them:
I've recently been testing Resin 1.1b4's performance compared with
mod_perl. The "benchmark.jar" from your web site included perl scripts that
really didn't take full advantage of the capabilities that mod_perl offers.
So, I wrote three perl modules that attempt to be similar to examples
included in the Resin 1.1b4 distribution and better reflect what mod_perl
can do.
Attached is a gzipped tar file with these perl modules, and you're free to
do whatever you want with them.
Overall, I've found Resin to be a very good product, yet ran into a few
problems during testing. These may be caused by the IBM 1.1.8 JDK rather
than the Resin package itself. For example when running Resin as a
stand-alone httpd server, then using "httperf" to benchmark it, the entire
JVM would freeze up when the --close-with-reset option was given to
httperf.
The JVM used in testing is IBM 1.1.8, on Red Hat Linux 6.0, 2x500MHz
Pentium III, 512MB RAM, Oracle 8.0.5 client libraries (JDBC). Apache 1.3.9
with DSO support was used for "Resin+Apache", with 60 max clients. A recent
version of mod_perl and Apache (without DSO) was used for the
"mod_perl+Apache" tests, also with 60 max clients. Finally, "Resin httpd"
is Resin running its own web server, with max threads set to 60.
These tests used httperf with persistent connections, making either 1000
or 100 requests per connection, depending on the test. The results on
"hello" are astounding:
Test: "hello" -- displays "Hello, World"
Size: 450 bytes
httperf: 40 concurrent connections, 1000 requests/connection
Results:
mod_perl+Apache: 1600 req/sec
Resin+Apache: 500 req/sec
Resin httpd: 4600 req/sec
Test: "bighello" -- lots of HTML + "Hello World"
Size: 23888 bytes
httperf: 40 concurrent connections, 100 requests/connection
Results:
mod_perl+Apache: 480 req/sec
Resin+Apache:300 req/sec
Resin httpd: 280 req/sec
Test: "database" -- query: "select NAME from DIVISION", returns 11 rows
Size: 460 bytes
httperf: 40 concurrent connections, 100 requests/connection
Results:
mod_perl+Apache: 570 req/sec
Resin+Apache:300 req/sec
Resin httpd: 450 req/sec
So, although "Resin httpd" performs very well with many small transfers
("hello"), the performance relative to mod_perl appears to degrade with
larger transfers ("bighello"). The database performance improved nicely
from where it was under Resin+Apache, but mod_perl still has the edge
there.
I also tested the "file.jsp", which I renamed "fortune" in the perl tests.
With http_load, the results again show mod_perl ahead:
Resin:
1584 fetches, 10 max parallel, 489610 bytes, in 10 seconds
309.097 mean bytes/connection
158.4 fetches/sec, 48961 bytes/sec
mod_perl:
6190 fetches, 10 max parallel, 1.98814e+06 bytes, in 10 seconds
321.186 mean bytes/connection
619 fetches/sec, 198814 bytes/sec
I'm not trying to belittle Resin in any way -- in fact I'm impressed by
both its design and performance. In learning about Resin during these
tests, I found that JSP is in many ways easier to use than mod_perl. The
smart caching that Resin does (with compiling .java - .class, etc.) is
effective, and the XML configuration is a joy to deal with compared to
Apache's httpd.conf.
So, it's nice, but not as fast as mod_perl on Linux. If anyone wants to
try this on another OS, I can supply a tar of the files we used for
testing.
- Perrin
Re: Caucho faster than mod_perl?
Bill Moseley wrote: Is Apache::Registry really that much of a hit in performance over a plain old Apache content handler? And if so, why? No, it isn't. This is a "Hello World" benchmark we're talking about here, designed to show the difference in startup costs. A real app would not have such a big difference in performance, although there would be some difference. Regardless, I encourage people to code for the Apache API rather than Apache::Registry. Besides the performance, it's just cleaner, and it avoids some problems that can happen when you turn your program into a sub, like accidental closures. - Perrin
Re: XML Configuration [Was: Re: Caucho faster than mod_perl?]
Matt Sergeant wrote: Also, what's different between Resin's smart caching and mod_perl's? Is it just like StatINC? It can cache output in a way similar to Mason. They call this "Smart Caching". It reloads servlet classes pretty cleanly when code is updated, but doesn't track dependencies, so it may be not as good as StatINC there. It also does a very nice job with JSP pages, turning them into first-class servlets. This produces less overhead than the current crop of perl-in-HTML tools available for mod_perl. We tested the same "Hello World" app as a servlet and as a JSP page and found no difference in performance. - Perrin
Re: Mod_Perl learning track
On Tue, 8 Feb 2000, Scott Chapman wrote: What magazines are good on Perl? At least two of the members on this list write for Web Techniques (Lincoln and Randal). And of course The Perl Journal is good. - Perrin
Re: Apache::ASP and Apache::SSI problems
On Thu, 10 Feb 2000 [EMAIL PROTECTED] wrote: Okay, I installed mod_perl, Apache::ASP and required modules like Apache::Filter, and Apache::SSI. (Note, some of the Apache:: modules will not isntall via CPAN.pm, they look for HTTP in some messed up directory where it oviously doesn't exist, you have to run make install manually or force the install through cpan which I have NO IDEA how to do). From the CPAN shell: force install Apache::SSI All the Ken Williams modules have this problem in their test scripts. Any chance you could fix that Ken? - Perrin
Re: What's the benefits of using XML ?
On Thu, 10 Feb 2000, Vlad Safronov wrote: What's the benefits of using XML in building web site with dynamic content? (web site is a front end to database) This is off-topic for this list, but I can't resist... What's the benefit of using XML? You get to buy expensive application servers! No, that can't be it... You get to buy a bunch of books on XML! No, that doesn't sound right either... It's an excuse to learn Java! No, somebody went and wrote a Perl parser... It's a way to store structured information like a database system, but without all that pesky concurrent access management and query capability! No, there must be something... Well, it's from Microsoft, so it must be great! Right? Okay, I admit there are places where XML is useful. Any place you used to say "comma separated values" can be s/// with XML and improve the readability of your data. It makes nice-looking config files too. It's very handy when you need a format for passing complex data in an OS and language independent way. It's also handy when you don't know what your data structure will be before you start, so you can't make a real database schema for it. Witness the folks at http://www.allaire.com/ who built an entire product (Spectra) around serializing objects in a single field as XML. Of course, if you only need to manipulate data using perl, you could just serialize with Storable and save some space/cycles. It can also be used when you want to store some structured data but you can't run a real database for some reason. It definitely beats making up your own addition to the million or so text-based data formats out there. Some people like to pull data from a database, turn it into XML as an intermediary format, and then use something like XSLT to turn it into HTML. Personally, I don't see a good reason to do this instead of just using perl's internal data strutcures for your intermediary format and using something like Template Toolkit (see CPAN) to format it into HTML. But, it's a matter of taste, and also of portability to other languages. Looks I came up with some reasons after all. So go ahead and use it if you see a fit. Just don't believe anything that a person with a product to sell tells you about XML. Or anything else. But that's a different story. - Perrin
Re: Apache::ASP and Apache::SSI problems
On Thu, 10 Feb 2000, Ken Williams wrote:
Yeah, I just saw the message. Since I tend not to use CPAN.pm (I like to
tinker around with stuff when I download/install) I've been ignorant of its
procedures.
The main thing to know is that it won't install if the tests fail, unless
you force it.
How about this: for Apache::AuthCookie I've been writing a system that will ask
the user at "perl Makefile.PL" time for the path to an httpd. The default is
$ENV{APACHE} or '/usr/lib/httpd/httpd'.
Will that work?
That would be great.
- Perrin
Re: What's the benefits of using XML ?
On Fri, 11 Feb 2000, Matt Sergeant wrote: XML and XSLT can provide this. Rather than write pages to a specific style with toolbars in the right place, and format things how I want them, I can write in XML, and down transform to HTML using a stylesheet. When I want to change the look of my site I change the stylesheet and the site changes with it. This isn't magic - a lot of template systems exist today - many of them written right here for mod_perl. But XSLT allows me to leverage those XML skills again. And I think authoring XML is easier than most of those template tools (although XSLT isn't trivial). Just a small plug for one of my favorite modules: Template Toolkit is very easy to use, and a couple of people have written plug-ins for it that handle arbitrary XML. If you're working on a project where you need to turn XML into HTML and want non-programmers to write and maintain the HTML templates, they may find it easier than XSLT. Of course, it's a Perl-only solution. - Perrin
Re: Apache::Session, DBD::Oracle
On Wed, 9 Feb 2000, Robert Locke wrote: We've been using the latest Apache::Session::DBI with some success using Oracle 8i (DBD::Oracle 1.03) as the data store. (Basically, we applied Ajit Deshpande's recommendation of patching DBIStore.pm with a large number for LongReadLen. See http:[EMAIL PROTECTED] for reference.) [...] To make a long story short, it seems that storing and then fetching a binary string is broken, but I'm not sure if this is an Oracle or DBD issue, or if I'm just doing something plain silly. It's my understanding that Oracle doesn't want you to use Long anymore. They prefer BLOB for this and are phasing out Long. Maybe they broke Long in the 8i release for certain situations. BLOB works fine from DBD::Oracle, so give it a try. You may need to set the type on the bind parameter to ORA_BLOB. perldoc DBD::Oracle for the scoop. - Perrin
Re: ORA-12154: TNS:could not resolve service name with DBI under Apache
On Wed, 29 Mar 2000, Jerome MOUREAUX wrote: Alias /indicators2/perl "/disc1/sherpa_a/indicators2/perl" [...] PerlSetEnv ORACLE_HOME /disc1/sherpa/oracle Do you really have a /disc1/sherpa directory and a /disc1/sherpa_a directory? - Perrin
Re: ORA-12154: TNS:could not resolve service name with DBI under Apache
On Wed, 29 Mar 2000, Jerome MOUREAUX wrote: My scripts run well if I launch it from the command line. Have you tried running them from the command line as the user who the webserver runs as? There may be something in your user environment that allows them to work which is not set up for this other user. - Perrin
Re: performance mongers: since when is using CGI.pm or Apache::Registry dishonorable?
On Fri, 31 Mar 2000, Vivek Khera wrote: My question to all of you who use handlers directly, how do you manage all your handler mappings? I've seen it done where you add a Location mapping for each handler you use, which corresponds to each "program" you need. This, in my experience, tends to be error prone and results in much cruft in your httpd.conf file. Works great for me. I don't tend to have hundreds of these, because I usually wrap groups of related functionality into one program and then use additional URI info to determine the exact action of each request - pretty typical multi-page CGI stuff. I like the freedom of mapping URIs that make sense to end users to modules in my class hierarchy that make sense to programmers. If you like the convenience of mapping directly to files from the URI, maybe a quick hack to treat the files as modules is in order. It could leave out the part that wraps your code in a sub and slaps a package on it, and just do a simple require instead (and maybe stat the file first and remove it from %INC if it has changed, although you could just use Randal's module for that). That would fix the things I don't like about Registry. As for making changes without restarting, I always restart my production servers when I change code, in order to maximize shared memory, but I can do this because I have a cluster of servers. I could see this being a problem if I only had one machine. - Perrin
Re: Transitioning from Apache::Registry
On Mon, 3 Apr 2000, Drew Taylor wrote:
In light of the recent "performance monger" thread, I am interested in
information about transitioning from Registry scripts to Apache
handlers. Here is my situation, which I'm guessing is pretty common.
[...]
use Search;
use CGI;
my $q = new CGI;
my $action = $q-param('A');
my $object = Search-($CGI);
if ($action eq 'R') {
$object-searchResults;
} elsif ($action eq 'S') {
$object-Search;
}
That should be pretty easy to convert, since you already have your main
program logic off in a separate module.
Should I just use Apache::Request instead of CGI.pm? (I do use the
cookie checkbox/pulldown functionality often).
Use Apache::Request and Apache::Cookie (both in the libapreq
distribution). I don't use the sticky forms stuff in CGI.pm, so I don't
know if there's a handy replacement. Check CPAN or just roll your own.
Maybe you could make a subclass of Apache::Request that adds somemethods
for this.
If there are any tutorials out there, I'd love some links.
It really isn't difficult enough to merit a tutorial, in my opinion. It's
just ordinary perl module stuff.
- Perrin
Re: Transitioning from Apache::Registry
On Mon, 3 Apr 2000, Drew Taylor wrote: So I just write a small handler which does the same thing as the .pl script and add a Location /cgi-bin/search.pl directive to use this handler? Yes, that should work fine. And I would need to setup a Location directive for each function (Search, Cart, etc)? You probably want to set up one for each Apache::Registry script you have right now, unless you don't like the way they're structured and want to re-work it. - Perrin
Re: prepare_cached and Apache::DBI.
On Wed, 5 Apr 2000, Paul Sullivan wrote: When attempting to use prepare_cached along with Apache::DBI, it returns this error once it has ran through each of the apache children. [Wed Apr 5 ...] [error] prepare_cached(...) statement handle DBI::st=HASH(0x8296788) is still active at /home/... line ... You should only be getting that error if there is more data on that statement handle that you haven't read. You can take care of that by calling $sth-finish, or by reading the rest of the data. - Perrin
Re: [slightly OT] Problem with cookies
On Thu, 6 Apr 2000, Drew Taylor wrote:
I have a site which uses cookies for user tracking. If you go to
http://cloudstock.com/, the server is sending the cookie but the browser
is not accepting it ("warn before accepting cookie" is on). If I go to
http://www.cloudstock.com/ the cookie is sent accepted.
Does your Set-Cookie header include a path setting? Some browsers require
that.
- Perrin
Re: [slightly OT] Problem with cookies
Jim Winstead wrote: An important point is that although "Host:" wasn't required until HTTP/1.1, all of the common browsers have sent it with 1.0 requests for some time. Yes, but I've had problems with corporate proxy servers that don't send it. - Perrin
Re: Apache::Session -- Go Out of Scope!!!
On Fri, 7 Apr 2000, Russell D. Weiss wrote: Normally, this works great. No problem at all. This object goes out of scope at the end of the script (it's scoped lexically with "my"). It also goes out of scope when "die" is explicitly called. If I add "die 'Blah blah blah'" to an app, things are fun. BUT... sometimes when there's a syntax error in the code, this object just stays in scope. This causes a real problem, because then the developer will just re-load. The process will sit there forever, waiting for the session lock to go away. But it never does... You could try using a PerlCleanupHandler to kill any open locks. $r-register_cleanup( \clear_locks ); - Perrin
Re: REDIRECT missing Cookie
Jim Winstead wrote:
On Apr 08, Zeqing Xia wrote:
Hi,
I'm having a problem with setting the cookie in a REDIRECT. Basically,
I'm doing this in a handler:
$r-headers_out-add('Set-Cookie' = $cookie);
$r-headers_out-add('Location' = $location);
return REDIRECT;
The $location is hit but the cookie is missing when I debug the handler
for $location.
I have tried various remedies, such as no_cache(),
unset('Content-length'), etc. But nothing seems have effect.
Can anyone point out what is the problem? Thanks so much.
This is a misfeature of some browsers. (Particular versions of
Internet Explorer, if I remember correctly.)
Some people have reported success with some browsers depending on
the order the headers appear, but the most bullet-proof solution
is to do a redirect with a header or meta-tag refresh instead of
an redirect. (Which is definitely not as user-friendly, but works
for all cookie-enabled browsers that I've come across.)
So you'd want to do something like:
$r-headers_out-add('Set-Cookie' = $cookie);
$r-headers_out-add('Refresh' = "0; url=$location");
$r-print("Sending you a href='$location'here/a.");
return OK;
Jim
You could also put the cookie info in the URL you redirect to (as a
query string), and make your handler look for it there if the cookie is
missing.
- Perrin
Re: Modperl/Apache deficiencies... Memory usage.
Each process of apache has it's registry which holds the compiled perl scripts in..., a copy of each for each process. This has become an issue for one of the companies that I work for, and I noted from monitoring the list that some people have apache processes that are upwards of 25Megs, which is frankly ridiculous. I have processes that large, but more than 50% of that is shared through copy-on-write. I wrote a very small perl engine for phhttpd that worked within it's threaded paradigm that sucked up a neglibible amount of memory which used a very basic version of Apache's registry. Can you explain how this uses less memory than mod_perl doing the same thing? Was it just that you were using fewer perl interpreters? If so, you need to improve your use of apache with a multi-server setup. The only way I could see phttpd really using less memory to do the same work is if you somehow managed to get perl to share more of its internals in memory. Did you? What I'm thinking is essentially we take the perl engine which has the apache registry and all the perl symbols etc., and seperate it into it's own process which would could be multithreaded (via pthreads) for multiple processor boxes. (above 2 this would be beneficial probably) On the front side the apache module API would just connect into this other process via shared memory pages (shmget et. al), or Unix pipes or something like that. This is how FastCGI, and all the Java servlet runners (JServ, Resin, etc.) work. The thing is, even if you run the perl interpreters in a multi-threaded process, it still needs one interpreter per perl thread and I don't know how much you'd be able to share between them. It might not be any smaller at all. My suggestion would be to look at the two-server approach for mod_perl, and if that doesn't work for you look at FastCGI, and if that doesn't work for you join the effort to get mod_perl working on Apache 2.0 with a multi-threaded model. Or just skip the preliminaries and go straight for the hack value... - Perrin
Re: Modperl/Apache deficiencies... Memory usage.
[EMAIL PROTECTED] wrote: Now with modperl the Perl garbage collector is NEVER used. Because the reference count of those variables is never decremented... it's because it's all in the registry, and it's hard to tell... hmm... what should I throw away, and what should I keep? ;-). What I know about Perl internals could fit on the head of a pin, but this strikes me as a very odd statement. If the garbage collector is never used, why do my lexical variables go out of scope and get destroyed? There are mod_perl packages like Apache::Session that absolutely depend on garbage collection of lexical variables to work. Are you saying that destroying the variables and actually reclaiming the memory are separate, and only the first is happening? - Perrin
Re: Modperl/Apache deficiencies... Memory usage.
[EMAIL PROTECTED] wrote:
my $i=0;
dosomefunnierstuff();
sub dosomefunnierstuff {
my $funnierstuff;
if($funnierstuff=~/funnier/) {
dosomefunnierstuff();
} else {
$funnierstuff="funnier".$i++;
}
print "Funnierstuff is $funnierstuff\n";
}
That proves the point a bit more clearly. It will show that each
layer of the "stack" keeps its own discreet copy of the variable.
Oh, I see what you're talking about. That's a closure. It's a language
feature, so changing that behavior would be significant. This shouldn't
be a problem if you simply avoid using closures in a recursive
algorithm. In your example, I believe only the value of $i will be
saved each time, since $funnierstuff will go out of scope at the end of
the block and get garbage collected.
- Perrin
Re: Problem with CGI::Carp under mod_perl
On Tue, 18 Apr 2000, Steve Hay wrote: I'm having problems using "CGI::Carp qw(fatalsToBrowser);" in modperl scripts. [...] The first one calls die() itself. Under Apache/CGI the die() message appears in the web browser (albeit preceded by a spurious Content-Type line), but under Apache/modperl the message goes to the error.log and a bizarre message appears in the web browser consisting of some output from the script, followed by a "200 OK" HTTP header, followed by a message suggesting that all was not OK after all (all the same as if CGI::Carp was not being used). Sounds like a difference in the way CGI scripts and mod_perl buffer. I fyou really want CGI::Carp to work, you need to make sure you don't send any output before it gets called. Maybe you have PerlSendHeader on? The second one has a syntax error. Under Apache/CGI a message about a compilation error appears in the web browser (but not the detail of the syntax error itself, which disappears completely - not even going to error.log!); under Apache/modperl an "Internal Server Error" message appears in the web browser (again just like CGI::Carp was not being used) and (curiously) the detail of the syntax error does now at least appear in error.log! CGI::Carp can't catch compile errors. - Perrin
Re: Problem with CGI::Carp under mod_perl
On Wed, 19 Apr 2000, Steve Hay wrote:
Sounds like a difference in the way CGI scripts and mod_perl buffer. I
fyou really want CGI::Carp to work, you need to make sure you don't send
any output before it gets called. Maybe you have PerlSendHeader on?
I did have PerlSendHeader On: turning it off causes a DOS Prompt to pop up
briefly, the die() message still goes to error.log and no output appears in
the browser at all!
I've never used mod_perl on Windows, so I don't know what causes the DOS
prompt. Keep PerlSendHeader On, but suppress your output until the script
has completed (i.e. collect it in a scalar and then print it all at once).
CGI::Carp can't help you if you've already sent a bunch of stuff to the
browser. You don't have this problem with mod_cgi because it buffers
everything until the script finishes.
CGI::Carp can't catch compile errors.
Oh. The CGI::Carp manpage says (regarding the use of "set_message()" to
control the output message): "In order to correctly intercept compile-time
errors, you should call set_message() from within a BEGIN{} block. ", which
suggests it does catch compile errors? I've tried both with "BEGIN
{set_message('...')}" and without a set_message() at all, but to no avail.
My mistake. CGI::Carp doesn't catch compile errors inside an eval block,
which is where you are when compiling under Apache::Registry. You could
work around this by commenting out the first line of the die() subroutine
where it checks to see if it's in an eval, but that's pretty lame and
makes CGI::Carp incompatible with use of eval blocks. I don't use
Apache::Registry or CGI::Carp, so I'm not very motivated to find a better
fix. You could contact the author.
- Perrin
Re: Apache::DBI disconnect?
"John S. Evans" wrote: Weird. The whole point of Apache::DBI (or so I understand it) is so that your $dbh stays valid across CGI or Handler calls. That's right. The disconnect call is a no-op when using Apache::DBI. I can only think of two reasons why I get the error message: 1) My child process is dying, and the DBI code is telling me that I never called disconnect() on that $dbh. I don't think this is the case, since the line (#119) in Apache::DBI that is referenced in the error is in the connect() call. 2) My $dbh has timed out, and Apache::DBI is attempting to reconnect. The code author decided not to perform a disconnect() because they knew the connection was already timed out. Those are both good guesses. I'd say the latter as well. One of your db handles probably failed to ping and got removed. I wouldn't worry about it. - Perrin
Re: Modperl/Apache deficiencies... Memory usage.
On Tue, 25 Apr 2000 [EMAIL PROTECTED] wrote:
With mod_proxy you really only need a few mod_perl processes because
no longer is the mod_perl ("heavy") apache process i/o bound. It's
now CPU bound. (or should be under heavy load)
I think for most of us this is usually not the case, since most web apps
involve using some kind of external data source like a database or search
engine. They spend most of their time waiting on that resource rather
than using the CPU.
Isn't is common wisdom that parallel processing is better for servers than
sequenential anyway, since it means most people don't have to wait as long
for a response? The sequential model is great if you're the next in line,
but terrible if there are 50 big requests in front of you and yours is
very small. Parallelism evens things out.
- Perrin
Re: Modperl/Apache deficiencies... Memory usage.
On Wed, 26 Apr 2000, Leslie Mikesell wrote:
According to Perrin Harkins:
On Tue, 25 Apr 2000 [EMAIL PROTECTED] wrote:
With mod_proxy you really only need a few mod_perl processes because
no longer is the mod_perl ("heavy") apache process i/o bound. It's
now CPU bound. (or should be under heavy load)
I think for most of us this is usually not the case, since most web apps
involve using some kind of external data source like a database or search
engine. They spend most of their time waiting on that resource rather
than using the CPU.
If you have tried it and it didn't work for you, please post the
details to help us understand your real bottleneck.
Sorry, I guess it was unclear what I was talking about. The proxy model
does work for me, but my mod_perl processes are not CPU bound because they
are still waiting for external data sources. I suspect most people are in
a similar situation.
The sequential model is great if you're the next in line,
but terrible if there are 50 big requests in front of you and yours is
very small. Parallelism evens things out.
Or it just adds more overhead. If you have enough parallelism to
keep your bottleneck busy, the 50th request can only come out slower
by switching among jobs more often.
Everything gets slowed down some by context switching, but my point is
that a long request doesn't hold up short requests in the same way if you
are processing in parallel. Also, if all of your requests are n seconds
long, and you process 10 of them that arrive at the server simultaneously,
the parallel model makes the last request finish about the same time as
the first, as opposed to finishing 9 * n later than the first.
- Perrin
Re: [RFC] modproxy:modperl ratios...
On Thu, 27 Apr 2000, Matt Sergeant wrote: Is there any benefit of mod_proxy over a real proxy front end like "Oops"? There's a big study of proxy servers posted at http://bakeoff.ircache.net/N02/. There are some expensive ones with dedicated hardware that perform well. Of course, there are tremendous benefits to having the source when you discover that the way some header is handled is not quite right for you, etc. For example, we found we wanted to disable the feature that gets a fresh page from the backend server when the user hits reload, since the cached copy in mod_proxy is the freshest that end users are allowed to see. Basically, mod_proxy and squid have had more work put into making them work as reverse proxies than other servers have. It doesn't like Oops supports a reverse proxy mode. It may not be immediately obvious, but you often need things like ProxyPassReverse (http://www.apache.org/docs/mod/mod_proxy.html#proxypassreverse) when you start using a proxy server in this way. - Perrin
Re: OOP, Inheritance, and mod_perl
On Thu, 27 Apr 2000, bahwi wrote: I have a question about OOP, Inheritance, and mod_perl. I have OO pre-loaded module A, which clients B, C, and D use. Each one get's their own, and the configuration variables are different. My question here is how much memory does this take up? Is the module's memory copied several times for clients B, C, and D? From what I have read I'm pretty sure that it's not that, just the variables are different, but I'd like to be sure. With the caveat that I've never read the Perl source, I'm pretty sure your compiled code is kept in memory separately from your variables, so you will only have one copy of the code. On to my second question, client E, F, and G each are using pre-loaded OO module A, but they are iheriting the class, make some changes, and then creating it with the new method. Each one has a different inheritance, and different methods are changed or added. How does this affect memory? Does each one get a full copy of the module in memory, and then it is changed? Again, there is only one copy of the code, but any variables are separate. - Perrin
Re: speed up/load balancing of session-based sites
On Thu, 27 Apr 2000, Dan McCormick wrote: If you split things between a proxy and a mod_perl server, the first hit would have to go through to the mod_perl server to initiate the session, but subsequent requests which may not need the session info could be sent to the proxy. Is that possible? Anything that actually uses the session data to do something specific to this user will have to go through to the backend mod_perl server. Generic pages without user customization could be served from the proxy cache though. Further, what are the standard ways to load balance a session-tracking app across multiple servers when the sessions are stored in memory and a given user has to be consistently sent back to the same machine? Most of the commercial load-balancing tools support the concept of "sticky" load-balancing, which routes people back to the same box each time. If you can't do that, you have to share the sessions across all app servers, probably by putting them into a database. Can round-robin DNS be counted on to send people back to the same server over the life of a given session? It should work, but round-robin DNS has its own shortcomings (no failover support) that usually send people to NAT-based approaches for high-traffic commercial sites. - Perrin
Re: growing processes
On Mon, 8 May 2000, Wim Kerkhoff wrote: On a fresh restart of apache, my processes are about 20 ~ 25 MB each, which is about normal for mod_perl (as far as I know). However, within a few hours (with little use except by our development team), the size is up to 40MB, and by the end of the day up to 60, 70, even 90MB each. You're probably doing something that is causing certain variables to have temporarily large values. As always, start with the guide: http://perl.apache.org/guide/performance.html#Memory_leakage You should also make sure you're doing the usual pre-loading and other suggestions from this section of the guide. You may find the section in the camel book on optimizing for size useful as well. - Perrin
Re: Am I Defeating The Purpose?
On Mon, 8 May 2000, Pierre J. Nicolas wrote: I've been tyring for the past several days to stop the infamous "Undefined Subroutine" messages. I implemented "solution 2" by adding a "require" statement in all of my scripts. That still didn't do it. Why do you have this problem in the first place? Is it because you have to use some libraries written in perl4 style, without packages? However, I just started using "do" and everything is working fine now. I know that "do" unconditionaly re-loads and re-compiles the perl script, but I wondering does a script still benefit from "mod_perl" if it uses "do" to load the perl libraries that it uses. You always beenfit from using mod_perl over CGI, because you avoid forking perl. However, the speedup from keeping the compiled code in memory is pretty big, and well worth the trouble on a busy site. Also, are the "do" scripts reloaded for each request or is it only once per unique script invocation. They're re-loaded on each request, which is what makes "do" so painful. If you really can't change your libraries to use packages (and export symbols if necessary), you could at least put the do statement inside a BEGIN block so that it will only happen once per script. You still waste memory by loading the same library multiple times, but you do get the performance benefits. - Perrin
Re: speed up/load balancing of session-based sites
Autarch wrote: On Mon, 8 May 2000, Perrin Harkins wrote: Some apps that use Apache::Session, like Embperl and Mason, have chosen to rely on cookies. They implement the cookie part themselves. Apache::Session has nothing to do with cookies. I don't know about Embperl but Mason a) doesn't do anything with Apache::Session at all if you don't make it do it yourself; and b) doesn't require you to use cookies if you do. Yeah, I know, I was just oversimplifying a bit. I should have said the example session-handling code that comes with Mason uses cookies. - Perrin
Re: speed up/load balancing of session-based sites
On Tue, 9 May 2000, Jeffrey W. Baker wrote: If you are using an RDBMS which has atomic operations, you can turn off locking in Apache::Session with no effect. I think every RDBMS I've seen, includig MySQL, guarantees atomicity at this level. On the subject of locking, I think that the daemon locker is going to be the fastest solution for a large-scale operation. Currently, the semaphore locker is the slowest (and also only scales to one machine), and the file locker is only slightly better. I saw in your previous announcement that you plan to support Berkeley DB 3 as a storage mechanism in your 1.5 release. I've found the automatic locking it does to be very efficient. For single machine setups I suspect it will outperform the other options. - Perrin
Re: growing processes
On Tue, 9 May 2000, Wim Kerkhoff wrote: On 09-May-2000 Perrin Harkins wrote: On Mon, 8 May 2000, Wim Kerkhoff wrote: On a fresh restart of apache, my processes are about 20 ~ 25 MB each, which is about normal for mod_perl (as far as I know). However, within a few hours (with little use except by our development team), the size is up to 40MB, and by the end of the day up to 60, 70, even 90MB each. You're probably doing something that is causing certain variables to have temporarily large values. As always, start with the guide: http://perl.apache.org/guide/performance.html#Memory_leakage You should also make sure you're doing the usual pre-loading and other suggestions from this section of the guide. You may find the section in the camel book on optimizing for size useful as well. - Perrin Ok, I'm going to go check that out right away. Could this have anything to do with it? [Tue May 9 09:39:50 2000] [error] DBI-connect failed: ORA-00604: error occurre d at recursive SQL level 1 ORA-00018: maximum number of sessions exceeded (DBD ERROR: OCISessionBegin) at / var/rubberoven.com/lib/nmSql.pm line 61 We're using SqlNet to connect multiple Linux web servers to Oracle running on a Solaris box. Nope, that's a different problem. - Perrin
Re: speed up/load balancing of session-based sites
On Tue, 9 May 2000, Jeremy Howard wrote: Murali said: As I understand from this discussion we have 2 methods involving creating a session-server which will store all session data. a) NFS mount a server which will store all session data b) Have a DB in this server which stores this data. Through a network connect to the DB and retrieve the info. I don't think you have to take either approach to store all session specific data. When a session is started, simply use a redirect a go to a URL that includes the IP address of the machine that will serve this session. From then on, the session will always go to the same machine, and therefore the session data doesn't have to be centralised or distributed. Unfortunately, that's not a high-availability solution. If the machine you sent that user to crashes, the user's session will be lost. Most people who have enough traffic to need a cluster of machines also care about sessions surviving individual machine problems. - Perrin
Re: Best approach for loading several modules
On Thu, 11 May 2000, Martin Wood wrote: Our Apache::Registry CGIs need access to a dozen or so core modules - is there an elegant solution to loading these without seeing a dozen or so use statements at the head of the script? Yes. As Vivek pointed out, you can move them all into a startup.pl that you pull in from httpd.conf with PerlRequire in your httpd.conf. If you don't need to import subroutines fom the modules (which ideally you wouldn't, since it wastes some memory) you can then just remove the use calls from the individual scripts. On the other hand, if you must have the use statements because the modules need to export some subs into the namespace of each script, you could use the C preprocessor on your perl code. Take a look at this: http://theoryx5.uwinnipeg.ca/CPAN/data/Filter/Filter/cpp.html Of course, you can also just continue to use "do common.pl", but put it in BEGIN block so it doesn't run on every request. - Perrin
Re: KHTTPD for static and Apache/mod_perl for dynamic??
On Mon, 15 May 2000, raptor wrote: My question is instead of using two Apache servers OR Apache+proxy scenario (static vs dynamic pages) will it be possible to leave static pages to kHTTPD and run only one APACHE server. It will, but you're missing part of the benefit from the proxy scenario. As the guide explains, running a proxy in front of your dynamic page server will help keep the number of processes down by dealing with slow clients for you. A super fast server for images and truly static pages is a good idea, although a simple stripped-down apache will probably handle more static page traffic than you can afford the bandwidth for anyway. - Perrin
Re: make test of Apache::SSI
On Mon, 15 May 2000, Wang, Pin-Chieh wrote: I am installing Apache::SSI under Mod_perl.. When I ran make test I have permission denied when it trying to start httpd , therefore the test failed. It doesn't work for me either. I think Ken was going to make this test optional. Anyway, if you're using CPAN to install (which I assume you are, because this wouldn't be a problem for you otherwise) you can just do a "force install". - Perrin
Re: make test of Apache::SSI
On Mon, 15 May 2000, Ken Williams wrote: I'd prefer to figure out why the test is failing, because most of the real testing is done using that test. But it's difficult because I've never seen the test fail. Looks like the problem lies in this line in t/real.t: my $HTTPD = "/home/ken/http/httpd"; This results in the following message, even if I supply a path to my httpd: t/real..Starting http server... /home/ken/http/httpd doesn't exist or isn't executable. Edit real.t if you want to test with a real apache server. Aborting real.t at real.t line 79. Shouldn't the path I give to the Makefile.PL override this one? By the way, this one does let me skip the test if I want to. Maybe it was Filter's test that didn't have an opt-out. - Perrin
Re: Apache::DBI and autocommit
On Tue, 16 May 2000, William Deegan wrote: If autocommit is not set and a script exits the transaction will be rolled back. The question I have is when the database handle is re-used will the autocommit still be unset if the script that set it errors out? Yes, Apache::DBI doesn't touch your autocommit settings. - Perrin
Re: Guide search engine (was Re: multiple copies of a module)
I know I'm late to this party, but I thought I'd point out a couple of options: - The Search::InvertedIndex module on CPAN (uses dbm files, I think). - The DBIx::TextIndex module on CPAN (uses MySQL). - The WAIT module on CPAN (uses dbm files). - Glimpse: http://webglimpse.org/. - Swish++: http://www.best.com/~pjl/software/swish/ (no, it's not the same one apache.org is using). I've also had great success with htdig. Maybe I'll try spidering the guide with and see how it does. - Perrin
Re: Confusion on Apache::DBI
On Thu, 18 May 2000, Niral Trivedi wrote: Now, with Apache::DBI, we'll have one DBI handle per child process during the server startup. Now, let's say one child has started its processing and hasn't served any request yet. Now, first request comes in and it will look for DB handle, which is available and start processing it. Now, second request comes in for same child process and request for DBI handle, which is still serving first request. Niral, you're not understanding Apache's multi-process daemon approach. No child process will ever try to serve more than request at once. These are not multi-threaded processes. - Perrin
Re: LARGE PERL footprint
On Fri, 19 May 2000, David Larkin wrote: Can anyone help explain why PERL gives such a large memory footprint advise how to get around it. Your array might be smaller if you pre-extend it to the size you need (see perldata). You could also look at some of the sneaky bit vector modules on CPAN if your data is appropriate for this. And you could try switching between perl's malloc and your system's malloc, though this requires you to build a new version of perl. - Perrin
Re: speed up/load balancing of session-based sites - POT
On Sat, 13 May 2000, Greg Cope wrote: : Likewise with sessions. Even if you load balance across multiple machines : you don't need to access a session database on every request. Most load : balancing systems have something so they'll send the seme "session" : (typically ip address) to the same backend server for N seconds as long as : the backend server is up of course. : : You can then put a daemon on each backend server as a local buffer and : cache between the httpds and the session database. This daemon can even : load balance the session information over several databases if it's : needed. : : viola, a lot less "real time" lookups in the database needed and a lot : less direct connections. [...] But - is this model not transaction safe ? - It may save alot of DB use and hence be alot quicker - but what if the webserver "disappears" halfway through an IP based load ballanced "sesssion" - a potential source of failure - something which a load ballancer is supposed to help with.. Also if the loadballance goes south, and a secound takes over does it have the same IP based session info (probably not - but could be wrong ...) - hence another potential source of failure. You have to do a write-through cache, i.e. you must write to the database as well as the cache, but you only have to READ the database if you don't find the session in the cache. If you have a fair amount of read-only use of the session this can be a big win. - Perrin
Re: [newbie] Passing Session Object Twixt Handlers
On Mon, 22 May 2000 [EMAIL PROTECTED] wrote: It seems the Apache::Session::DBI isn't actually changing anything in the database, since next time I tie the session the only thing it has is the _session_id I tied it with in the first place. Keep in mind that Apache::Session only writes to the database when the object is destroyed. Make sure it is really going out of scope. - Perrin
Re: Apache::DBI
On Mon, 22 May 2000, Mark Holt wrote: Newbie to the list, apologies if this is a FAQ, but I checked the archives... I built a modperl-enabled server, (Apache/1.3.9 Ben-SSL/1.37 (Unix) mod_perl/1.21) and have been running it for some time. I have the mod_perl as a shared module, and I recently enabled it to start development. I have a startup script in operation and things work fine, until I try to preload any sort of DBI module. I tried Mysql, DBI::mysql, Apache::DBI, all to no avail. The server gives no error message, it simply acts as though it is starting, but does not show up in the process list. Is there something I'm missing? Are you pulling in the DBI and DBD modules in your startup.pl? Is there any message in the error_log? Apache::DBI is not at all like DBD::mysql, so it sounds like you may have a problem totally unrelated to DBI. - Perrin
Re: [Re: Questions about Apache::Session]
On 22 May 2000, Edgardo Szulsztein wrote: Now, I would like to make it work with the postgresql database server. How could I do it (I don't like the list, but I couldn't find this information in the documentation). See the Apache::iNcom::Session module on CPAN. It does exactly this. - Perrin
Re: DBD::Oracle Apache::DBI
On Mon, 22 May 2000, Ian Kallen wrote: I've done everything I can think of to shore up any DB connection flakiness but I'm still plagued by errors such as these: DBD::Oracle::db selectcol_arrayref failed: ORA-12571: TNS:packet writer failure ...this is only a problem under mod_perl, outside of the mod_perl/Apache::DBI environment everything seems fine. Once the db connection is in this state, it's useless until the server gets a restart. My connect strings look good and agree, I put Stas' ping method in the DBD::Oracle::db package, set a low timeout, called Oracle (they don't want to hear about it). Everything is the latest versions of mod_perl/Apache/DBI/DBD::Oracle connecting to an Oracle 8.1.5 db on Solaris. Is Apache::DBI not up to it? (it looks simple enough) Maybe there's a better persistent connection method I should be looking at? Apache::DBI is dirt simple, and it's the only method I know for persistent connections. You can just keep your $dbh as a global, but Apache::DBI is just barely more complex than that. Your problem is almost certainly coming from Oracle in one way or another. You may not see this happening with non mod_perl scripts because they don't live long enough or peform enough queries. One thing I've encountered is that when the ping method fails in a child process I can never get a successful reconnect from that same process again. I get around this by killing that process (Apache::exit()) after it sends out an error page. New child processes spawned by Apache don't have any trouble connecting. This is with Oracle 8 on Linux. - Perrin
Re: Apache::DBI
On Mon, 22 May 2000, Mark Holt wrote: I have tried pulling in *any* DBI or DBD module one at a time and ALL of them cause the server to fail to start. There is no error log message. Does DBI work for you from command-line scripts? Did the tests for DBD::mysql pass? Is there an order I must use them in? Apache::DBI installed without a hitch, are there other pitfalls of which I should be aware? Yes, the Apache::DBI docs explain that you must load it before any other DBI-related modules. - Perrin
Re: global variables and reparsing question (low priority ;)
I don't quite understand what you're trying to do, but what you have
here is a closure and it looks like you want a real global instead.
(man perlref if "closure" doesn't ring a bell.) Some of your language
makes it look like you may have some confusion between global and
lexicals. At any rate, "do" is nearly always a bad thing and the
business about being parsed twice only applies to Apache's config file
and Perl sections in it, not to your modules.
- Perrin
Marc Lehmann wrote:
While I understand how my problem happens, it just caught me again
(letting me debug another two hours), and so I wanted to ask why this
peculiar behaviour is really necessary.
But straight to the point. If I have a module like this:
=
package othermodule;
my $global = 5;
sub set_global {
$global = shift;
}
=
And use this from another module:
=
othermodule::set_global 7;
=
Then, to my surprise, _sometimes_ the $global will be set, and sometimes not.
The reason for this seems to be that othermodule is parsed twice: Once
during configuration parsing and once later. The first time it is parsed,
$global is created and bound to ste_global. Other modules that call the
function get a reference to it cimpiled in.
Then, when the file is parsed again, _another_ instance of $global gets
created, and another instance of set_global is compiled. Now there are two
versions of set_global, two versions of $global, and which one gets called
very much depends on random factors.
To my pity, _some_ global (non-lexical) variables also get cleared, but
not in all packages.
Now, since this is really hard to debug (the order of function declaration
within the same file and wether the funciton is exported or not, and
wether there are other callers outside the file etc...
To remedy this, many of my modules (the bad point is that even modules I have
no control over suffer from this problem) start with something like this:
=
if (!defined $PApp::Apache::compiled) { eval do { local $/; DATA };
die $@ if $@ } 1;
__DATA__
#line 6 "(PApp::Apache source)"
=
This mostly remedies the situation. The only "side effects" that I found
so far is that my httpds reuqire much _less_ memory and run _much_ more
stable (mod_perl really is very unstable for large applications). Apart
from expected semantics of global lexical variables.
So my very own problem is solvable (either by using package variables
or that ugly but working trick above). Third-party modules (e.g. from
CPAN) however, still suffer from this problem, and due to my dumbness I
fall into that trap again and again.
And so my question is: why does this behaviour exist, and why is it
necessary (the documents I saw so far only told me that this "has
something to do with apache's configuration file parsing", which doesn't
explain much, especially as it does seem unnecessary).
Re: global variables and reparsing (short reproducible example)
On Wed, 24 May 2000, Marc Lehmann wrote:
On Wed, May 24, 2000 at 12:52:37AM +0300, Stas Bekman [EMAIL PROTECTED] wrote:
You can control what's being reloaded and what's not:
http://perl.apache.org/guide/config.html#Apache_Restarts_Twice_On_Start
Not really. This would require intimate knowledge of some module's
implementation (which files it requires and which it doesn't). My hackish
example did something similar, indeed, but it had to be embedded into the
module source, which is somewhat inconvinient.
If you don't have PerlFreshRestart turned on (it is not recommended on
production servers), only the stuff in your config file (Perl sections
and PerlModule/PerlRequire) gets called twice. You can block things out
like this:
Perl
unless ($Apache::Server::ReStarting) {
...
}
/Perl
BTW, what is the connection between that part of config.html and
PerlFreshRestart? Can it happen that modules get reparsed even when
PerlFreshRestart is off?
Only the stuff in your config file should get reparsed.
Should I abstain from trying to restart apache when using any modules
requiring full perl semantics? And is SIGUSR1 safe?
People have reported some growth in memory consumption when using
SUGUSR1. Personally, I always do cold stop and start on my production
servers, in order to maximize shared memory, but this is difficult when
you don't have a load-balanced cluster at your disposal...
- Perrin
Re: global variables and reparsing (short reproducible example)
On Wed, 24 May 2000, Marc Lehmann wrote:
I was under the impression that you cannot configure Apache from a
PerlRequire. If that is not the case (and somehow works) I'd really like
to get away from perlsections.
You can only configure Apache from Perl sections, but you can load all
your modules, shared data, etc. from a file pulled in with PerlRequire.
Your sub x is a closure. That's why it returns the previous value of
$x. When it gets re-defined, it should start looking at the value of the
new $x.
You cannot redefine closures as well.
Closures are a confusing aspect of perl, and I could easily be mistaken
about them, but Randal and Doug seem to agree that this is a closure:
my $x;
sub show_x {
print $x;
}
The easiest definition I've found is the one that Damian Conway has in his
OO Perl book:
"In Perl, a closure is just a subroutine that refers to one or more
lexical variables declared outside the subroutine itself."
When this code gets parsed, the $x goes out of scope at the end of the
block, but show_x keeps a copy of it.
I think that this is the source of the behavior you saw where it looked
like there were multiple versions of subs and variables being created.
Anyway, I was just trying to help you solve your problem. There's no need
to get defensive.
- Perrin
Re: High-volume mod_perl based ecommerce sites?
On Thu, 25 May 2000, Barry Robison wrote: You may want to check out http://www.opensales.org/html/source.shtml, rather than starting from scratch .. I haven't used it, but it's a Perl based GPL commerce solution. Or you may not. It doesn't support mod_perl. I'd suggest looking at Tallyman or Yams instead. - Perrin
Re: CGI.pm broken with Apache/mod_perl/warnings? (was Re: XHTML_DTDredefined?)
On 25 May 2000, Randal L. Schwartz wrote:
I got no "no warnings". this is still 5.5.3 :)
There's always the time-honored
{
local $^W;
}
- Perrin
books (was High-volume mod_perl based ecommerce sites?)
On Thu, 25 May 2000, Neil Conway wrote: I'm probably a novice programmer, at least by the standards of most of the people on this list. I'm 16, and since I haven't taken Computer Science at university yet, I'm a bit lacking in 'formal programming education'. I'd rather not form bad habits - is there any advice anyone can give me on how to write, clean Perl (OO or otherwise)? Are there any good books I can pick up? Sure, try "Advanced Perl Programming" by Sriram Srinivasan and "Object Oriented Perl" by Damian Conway. For general advice, "Code Complete" by Steve McConnell is good. - Perrin
Re: ques on error msg
On Fri, 26 May 2000, amy wrote:
## second error
DBI::db=HASH(0x1ff304)-disconnect invalidates 1 active
statement handle (either destroy statement handles or call
finish on them before disconnecting) at
/usr/local/bin/apache/cgi-bin/lib.pl line 41.
##
I avoid this type of error by pushing a cleanup handler to take care of
it, like this:
sub finisher {
while ((my $key, my $sth) =
each %{$dbh-{'CachedKids'}}) {
$sth-finish();
}
}
I call finish on my statements explicitly when I can, but if something
exits in an irregular way (i.e. before I was able to call finsih) this
handler will take care of it.
Maybe this should be folded into Apache::DBI, like the automatic rollback?
- Perrin
Re: [ANNOUNCE] Apache::Session 1.51
On Fri, 26 May 2000, Jeffrey W. Baker wrote: I have released Apache::Session 1.51. The addition of the Oracle backing store took less time than expected. It is included and tested in this release. This is the only change from 1.50. http://www.cpan.org/modules/by-module/Apache/Apache-Session-1.51.tar.gz This new stuff looks really good. This module has come a long way in the time I've been watching the list. A couple of notes on the Oracle storage module: - Using "FOR UPDATE" forces the transactional lock model. Is it possible to make this optional? The other modes allow the use of a "enforce data integrity only" locking style which is sometimes needed. I'm less worried about people overwriting their own session data than I am about stale locks, although I suppose Apache::DBI's cleanup handler should take care of them. - Oracle (the company) says not to use LONG anymore. They are trying to move everything to CLOB/BLOB. I modified my Apache::Session::DBI to support this, which mostly involved specifying "ora_type = ORA_BLOB" in my bind variable parameters. Maybe we need and Oracle8 module, separate from the standard one? By the way, BLOBs don't work with synonyms, so you have to specify the schema name in the SQL when using them. I know you were working on a BerkeleyDB storage module. Are you still working on it, or did you throw up your hands in disgust over the BerkeleyDB module? Although I don't have time right now, I could eventually work on this if you aren't already doing it. Finally, everybody loves benchmarks. Do you have any cool speed comparisons between the various storage and locking options? I'm sure the list would be very interested. Even better would be a little benhcmarking script that people could use on their own systems to do comparisons. - Perrin
Re: [ANNOUNCE] Apache::Session 1.51
On Fri, 26 May 2000, Jeffrey W. Baker wrote: A couple of notes on the Oracle storage module: - Using "FOR UPDATE" forces the transactional lock model. Is it possible to make this optional? The other modes allow the use of a "enforce data integrity only" locking style which is sometimes needed. I'm less worried about people overwriting their own session data than I am about stale locks, although I suppose Apache::DBI's cleanup handler should take care of them. I assume that if people are using a transactional database, they are knowledgable about transactions. That's why I made the Commit argument mandatory for the Oracle and Postgres backing stores: it forces people to consider their transaction policy. My only complaint, and it's one I can easily solve by editing the source, is that I can't get the "no locks, just atomic updates" policy that I currently use in the new module, even if I use the Null locker. - Oracle (the company) says not to use LONG anymore. They are trying to move everything to CLOB/BLOB. I modified my Apache::Session::DBI to support this, which mostly involved specifying "ora_type = ORA_BLOB" in my bind variable parameters. Maybe we need and Oracle8 module, separate from the standard one? By the way, BLOBs don't work with synonyms, so you have to specify the schema name in the SQL when using them. That's lame. So, we would need to pass the schema name as an argument? Remind me again what's wrong with LONG? From my perspective, nothing at all. I think they even perform better than BLOB at the moment, at least through DBI. The problem is, Oracle is phasing them out and will eventually drop support for them entirely. They don't want to admit that BLOBs breaking synonyms is a bug either. - Perrin
Re: Changes to module code only take effect if perl program updated
On Wed, 31 May 2000, Karl Djemal wrote: I'm trying to get up to speed with mod_perl and have been playing around with Apache::StatINC so that any perl modules I update get reloaded. From what I can see in the error_log file it does get re-loaded, but the changes don't seem to take effect until I touch my perl program that 'uses' this module. Why? Does your Apache::Registry script import anything from this module? If not, it's possible that perl has in-lined your module's subroutines in the Registry script as an optimization during compiling. Those warnings you mentioned usually indicate that. - Perrin
Re: [OT] Bean::*?
On Wed, 31 May 2000, DeWitt Clinton wrote: My question is this -- has anyone written an implementation of the Java Bean standard in Perl? I don't think we need to have a major debate about the shortcomings of beans on the mod_perl list. But I am wondering if anyone has pulled this off yet. If we left out the event driven parts of the bean spec, and just focused on the property side of things, this could be a rather short project to get the core down. Maybe I'm just being thick, but if you leave out the event driven parts of the spec, what's left that isn't just an ordinary Perl class? You can easily make classes to hold data using Class::Struct or Class::MethodMaker. Is it just that you want a safer way to do introspection than looking at the data structure the class is made from? Maybe you want the Class::Fields module from CPAN. - Perrin
Re: Human readable flatfiles
I have a perl (non-modperl) program that needs some input data. Currently, it reads in the data by "require"ing another perl script that has statements to set the variables (as global variables). I did it this way so that I can easily edit the include file if I want to change values, and I can even put code in the include file to calculate values. But, I am finding that this does not work in modperl under "use strict". Apparently, code in a file called by require is not in the same variable scope. (If "use strict" is off, it works sometimes but other times the variables come out with the values from the previous invocation.) You need to read up a little on modules and "require" in Perl5. The quick and dirty solution is to use "do" instead of require. That will solve your immediate problem, but you'll still be reading the files every time which might eventually become a performance hit. What I do when I have things like config files is make actual unique packages of them. For example: (In My/Module.pm) package My::Module; # don't use strict here $foo = 7; (In some other program) use My::Module; print "$My::Module::foo\n"; Honestly though, your example makes it look like you'd be better off with dbm files and Tie::MLDBM or something. - Perrin
Re: Human readable flatfiles
Philip Mak wrote:
I can't seem to get "do" to work. I did this:
my $series_name;
do "series_$series.i"; # -- note include filename depends on a variable
print "$series_name\n";
Your lexical ("my") variable in the same scope is taking precedence, and
the "do' is not allowed to see lexicals in its enclosing scope. If you
make $series_name a global, I think it will work. The alternative is to
read the file yourself and eval it, which would allow it to see the
lexical.
I also tried the package thing:
What I do when I have things like config files is make actual unique
packages of them. For example:
(In My/Module.pm)
package My::Module;
# don't use strict here
$foo = 7;
(In some other program)
use My::Module;
print "$My::Module::foo\n";
How would this work if the include filename has to depend on a variable? I
think I have the parsing wrong; I kept getting error messages. e.g.
something like:
use $series;
print "$$series::series_name\n";
If it depends on a variable, replace the "use" with "require". The
"use" statment is evaluated at compile time, and "require" at run time.
Keep in mind that require only loads each file once, so if you need to
show changes in these files that happen while the server is running this
won't work for you.
Honestly though, your example makes it look like you'd be better off with
dbm files and Tie::MLDBM or something.
The database files would have to be human readable though, such that they
can be edited by a text editor. The data that I am making includes an
index to content that is maintained by other webmasters. The other
webmasters don't even know how to use a UNIX shell, so I have to keep it
simple for them. If I used a binary file format I'd have to make them
learn the tools for changing it.
Depending on how much of this data you have and how timely it has to be,
you could make a cron that crawls the text files every 5 minutes or so
and build a dbm with their data in it.
- Perrin
Re: Changes to module code only take effect if perl program updated
Karl Djemal wrote: Thanks for the reply. I'm not quite sure what you are asking here - may be what I asked wasn't to clear. I was asking if you import any subs or variables from MyModule.pm. If you don't know what I mean, then you probably don't. I'm using Apache::Registry as a PerlHandler to run my perl CGI script. This perl program in turn does a use MyModule. What I don't understand is that I can see my perl module 'MyModule.pm' get reloaded if I change it, but the changes are not picked up until I update my perl program. It is when I update my perl program I start to see the 'Subroutine... redefined..' warnings. Does any one know why the parent httpd process doesn't reload my module (could it be that it does, but doesn't record this in the log file?). It definitely sounds like inlined subroutines to me. When your Apache::Registry script got compiled, it inlined some subs from MyModule.pm for performance reasons. They won't get redefined until your Registry script gets compiled again. This isn't related to the parent process. Looks like you'll have to touch the file, or do some hacking on StatINC to make it unload your Registry script when it reloads your modules. - Perrin
Re: Apache::Session
On Thu, 1 Jun 2000, Niral Trivedi wrote: Now, only way to remove this information from backing store is to do 'tied(%session_hash)-delete', Am I right? I mean is there any way we can remove these entries by setting 'time_out_interval' or something like that?? Your cron job idea sounds fine. If you need the expiration to be more precise than that, you could alter the storage class you're using to check a timeout value in the session against the current time. If the session is too old, it acts like it couldn't find it. In the database classes this could be done as a WHERE clause in the SQL. You'd still want to run a cron to actually delete the old session data. - Perrin
Re: global variables and reparsing question (low priority ;)
On Thu, 1 Jun 2000, Marc Lehmann wrote: It's easy, I just have to kick my ass each time I want to use a lexical for data abstraction and use a package variable instead, with only the exception that I have to be very careful that I never re-use the same name. This is quite difficult for code that resides in many files (package variables are, of course, global to the whole package not just the file or the block), but not unsolvable. The simplest thing is to just not use PerlFreshRestart or Apache::StatINC (which works in a similar way). These are generally a bad idea for production servers anyway, due to their effect on shared memory, and mostly exist to ease development, although in your case they had the opposite effect. There are several warnings about mysterious problems caused by PerlFreshRestart in the guide, and my impression is that most people simply avoid it altogether. If you don't use these your code should work just fine, lexicals, accessors, and all. - Perrin
Re: FW: [Q] session mgmt
On Thu, 1 Jun 2000 [EMAIL PROTECTED] wrote: So my questions are: how to save session info from page 1 and allow a web server that services age 3 to access it. Have a look at Apache::Session. Use a database or files over NFS if you have a cluster. I've had better luck with databases than NFS. MS suggests to use their distributed directory services to store session info (instead of cookies) and they do not suggest to use a DB for performance reasons. I would like to know if there are alternatives to MS solutions. "Don't use a database, use OUR database." Ignore them, they're just trying to make money from you. Use a database. - Perrin
Re: [benchmark] DBI/preload (was Re: [RFC] improving memory mappingthru code exercising)
On Sat, 3 Jun 2000, Stas Bekman wrote:
* install_driver (2):
DBI-install_driver("mysql");
I've never seen that before, and it isn't in the DBI perldoc. Is it safer
than "use DBD::mysql;"?
- Perrin
Re: [benchmark] DBI/preload (was Re: [RFC] improving memory mappingthru code exercising)
On Sat, 3 Jun 2000, Stas Bekman wrote: correction for the 3rd version (had the wrong startup), but it's almost the same. Version Size SharedDiff Test type 1 3469312 2609152 860160 install_driver 2 3481600 2605056 876544 install_driver connect_on_init 3 3469312 2588672 880640 preload driver 4 3477504 2482176 995328 nothing added 5 3481600 2469888 1011712 connect_on_init Cool, thanks for running the test! I will put this information to good use...
Re: Data structure question
On Tue, 6 Jun 2000, Drew Taylor wrote: I know about tied hashes - Thanks Damien for your excellent book! - but there is a performance penalty. How big is this penalty? Is it worth using tied hashes? Versus an array of hash refs? They're a lot slower than normal data structures, or even normal object methods. Whether that slowness will be noticeable next to the slowness of accessing a database is questionable. There were a few benchmarks posted to this list that you could dig out of the archive. - Perrin
Re: [performance/benchmark] printing techniques
On Wed, 7 Jun 2000, Stas Bekman wrote: And the results are: single_print: 1 wallclock secs ( 1.74 usr + 0.05 sys = 1.79 CPU) here_print:3 wallclock secs ( 1.79 usr + 0.07 sys = 1.86 CPU) list_print:7 wallclock secs ( 6.57 usr + 0.01 sys = 6.58 CPU) multi_print: 10 wallclock secs (10.72 usr + 0.03 sys = 10.75 CPU) Never mind the performance, that multi_print and list_print are just U-G-L-Y! Here doc rules. Great for SQL too. - Perrin
[OT] Re: [performance/benchmark] printing techniques
On Wed, 7 Jun 2000, Matt Sergeant wrote:
On Wed, 7 Jun 2000, Eric Cholet wrote:
This said, i hurry back to s/"constant strings"/'constant strings'/g;
Those two are equal.
Yes, although it's counter-intutive there's no real performance hit
from double-quoting constant strings.
The one that bugs me is when I see people doing this:
$hash{"$key"}
instead of this:
$hash{$key}
That one is actually in the perlfaq man page, but I still see it all the
time. The performance difference is very small but it does exist, and you
can get unintended results from stringifying some things.
- Perrin
Re: Template techniques
On Thu, 8 Jun 2000, Bernhard Graf wrote: Chris Winters wrote: The newest version of Template Toolkit (currently in alpha) supports compiling templates to perl code. See about 2/3 of the way down the the README at www.template-toolkit.org. Why reinvent the wheel? :) Also the current stable (1.06) can do this. Not quite. The current version uses its own system of opcodes (!) which are implemented as closures. Compiling to perl code gives much better performance, which is why Andy is changing this. Template Toolkit rocks, and will rock even more when it has the extra speed. - Perrin
Re: Template techniques [ newbie alert + long ]
On Thu, 8 Jun 2000, Greg Cope wrote: My original question was not related to templates (I'll use embperl for that) Well, I'm confused now. You'll use Embperl for templates but you're not using Embperl for templates? - the area I was trying to explore was how to read a template (all HTML with a few !--TAGS-- in it) and the sub in the new content. Embperl would work fine for that, but it's overkill. Your substitution approach is slower than compiling to perl subs, especially since you have to load the file, but saves lots of memory and is fine for something as simple as this. Has anyone any suggestions as to speeding this up - yet keeping it simple - I have played with referances to avoid all the variable copying etc . ? Caching templates in memory would certainly help, but you'll eat up a chunk of RAM. - Perrin
Re: [OT] Re: [performance/benchmark] printing techniques
On Thu, 8 Jun 2000, Matt Sergeant wrote:
The one that bugs me is when I see people doing this:
$hash{"$key"}
instead of this:
$hash{$key}
Those two now also result in the same code. ;-)
But the former is just ugly.
Sometimes it's worse than just ugly. See the entry in the Perl FAQ:
http://www.perl.com/pub/doc/manual/html/pod/perlfaq4.html#What_s_wrong_with_always_quoting
Not likely that anyone would be using something as a hash key that would
suffer from being stringified, but possible. It's definitely a bit slower
as well, but that's below the noise level.
- Perrin
Re: Any suggestions or help, Thanks
Turn on Apache::DBI's debugging messages and see if it's working properly. - Perrin On Wed, 7 Jun 2000, Hui Zhu wrote: Hi Everybody: I got big problems. Same query and same script. Sometimes it works fine but sometimes i get the following errors (i am so frustrated, have no idea what i need to do): DBD::mysql::db do failed: MySQL server has gone away at cnznetmod.pm line 151. DBD::mysql::st execute failed: MySQL server has gone away at cnznetmod.pm line 159. DBD::mysql::st execute failed: MySQL server has gone away at cnznetmod.pm line 130. DBD::mysql::st fetchrow failed: fetch() without execute() at cnznetmod.pm line 131. I remember that i had a same problem when our system upgrated last time. Once i reinstalled ApacheDBI-0.87, it works fine. But this time, i did so but it still doesn't work. Thank you for your help. Steven. BTW: Linux redhat6.2, ApacheDBI-0.87, DBI-1.13, Modperl-1.21, Msql-Mysql-modules-1.22 Mysql3.22
Re: Template techniques [ newbie alert + long ]
On Thu, 8 Jun 2000, Greg Cope wrote: - the area I was trying to explore was how to read a template (all HTML with a few !--TAGS-- in it) and the sub in the new content. Embperl would work fine for that, but it's overkill. Your substitution approach is slower than compiling to perl subs, especially since you have to load the file, but saves lots of memory and is fine for something as simple as this. Can you enlighten me into the compiling to perl subs ? It's what Matt was talking about. Your program parses the template, generates perl code that produces the correct output, evals the code, and stores the results in a sub reference which you can call whenever you want that template. The first time I ever saw this done was with ePerl, but I don't know if that was really the first. All the embedded perl systems popular around here (Embperl, Apache::ASP, Mason, etc.) use some variation on this technique. I think the world's record for most compact implementation goes to Randal for a small post you can find in the archive here: http:[EMAIL PROTECTED] The file gets loaded once into shared memory - most (stripped) HTML files are only a few 10's of K. Also the file gets loaded once at startup - not during the request stage. You probably won't get much faster than that then, no matter what you do. Just make sure your regexps are fast (maybe use "study"?) and use references for passing data. - Perrin
Re: [OT now] Re: Template techniques
On Fri, 9 Jun 2000, Drew Taylor wrote: I really like the fact that templates can be compiled to perl code cached. Any others besides Mason EmbPerl (and TT in the near future)? Sure: Apache::ePerl, Apache::ASP, Text::Template, and about a million unreleased modules that people wrote for their own use. (I think writing a module that does this should be a rite of passage in Perl hacking.) - Perrin
Re: Template techniques
On Fri, 9 Jun 2000, Roger Espel Llima wrote: I'm developping yet another toolkit for templating under mod_perl (don't flame me YET, it does things that are significantly different from Mason, Embperl and others: namely completely separation of data and code, good multilingual support, and a reverse-include-based (aka OO without code) component model). Sounds like Template Toolkit to me. Or maybe even Apache::Taco (now defunct?) which worked by calling external functions. Have fun developing it, but think long and hard before you put another templating module on CPAN. So, if there's a serious argument that compiling to perl code is better, I'm interested. So far, I'm a bit doubtful that it's worth it, esp. for a large site (where you'd spend a lot of time on the perl compilation phase, reading perl files over and over; unless someone figures a way to store bytecode?). This being mod_perl, you only have to compile the perl code once per process (or once at startup). There is a situation when compiling to perl is usually worse. When you're just doing simple SSI-ish stuff, with lots of unique pages, turning every page into a perl sub becomes a memory hog which outweighs the speed benefit. That's when simpler techniques like the stuff used in Apache::SSI come out ahead. - Perrin
Apache::DProf tip
Here's something that might be obvious to others but took me a while to figure out: If you want to get useful profiling information, you need to initialize the debugger before your modules get compiled. If you pull in your modules from startup.pl, you can accomplish this by putting a block like this in your httpd.conf before the "PerlRequire startup.pl" statement: Perl use Apache::DProf; use Apache::DB; Apache::DB-init; /Perl The "PerlModule Apache::DProf" directove suggested in the Apache::DProf docs also seems to work, but be careful because load order seems to matter here. - Perrin
Re: Perl vs Java
On Sun, 11 Jun 2000, Matt Sergeant wrote: There are posts in the archive about this. Here's a quick summary: You can make Java slow. You can make mod_perl slow. Java (servlets, jsp, etc) can use a lot of memory. mod_perl can use a lot of memory. Servlets can be very fast. mod_perl can be very fast. I think that covers most of the arguments. This cracked me up! Thanks Matt. - Perrin
