Re: Template techniques
On Thu, 8 Jun 2000 [EMAIL PROTECTED] wrote: As far as I've seen, the fastest template systems are the ones that convert the template to Perl code. So that's what I do. The templates all call a method (in my case $Response-Write()) which appends to a string. If there are no exceptions (see the guide) the string is sent to the browser. If there are exceptions, I parse/send an error template with the error in the template. I'm curious Matt, as opposed to what?, reparsing the template each run? Clearly reparsing would be a big loser in terms of performance. As opposed to parsing into a tree and working from that. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: [performance/benchmark] printing techniques
On 8 Jun 2000, Stephen Zander wrote: As Matt has already commented, in the handler the method call overheads swamps all the other activities. so concat_print aggrlist_print (yes, method invocation in perl really is that bad). When you remove that overhead the extra OPs in aggrlist_print become the dominating factor. Perhaps it would be worth testing the horribly ugly: Apache::print($r, output); Rather than plain print(). -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: Method overhead benchmarks [Was: [performance/benchmark] printing techniques]
On Thu, 8 Jun 2000, Barrie Slaymaker wrote: Stephen Zander wrote: As Matt has already commented, in the handler the method call overheads swamps all the other activities. Just to clarify: that's only important if you are doing very few other activities, or if those other activities also include a high percentage of method calls: You also forgot that print() goes to a tied STDOUT, which is even more of an overhead... -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: Newbie Questions about Perl/Apache
On Wed, 7 Jun 2000, Rich Lemanski wrote: Hello all, I just joined the list the other day and have some questions about internal and external Perl interpretors for Apache. Forgive my lack of knowledge, but I am assuming that mod_perl is the Perl interpretor that runs internally to Apache and starts when Apache starts. The external Perl interpretor option, Perl CGI, is run externally and must be spawned by Apache anytime they are needed. Do I have the right idea? Almost, but not quite. Perl launched as a CGI only has access to environment variables and STDIN (in the case of POST requests) it has no access to any of the server's API. CGI is a cross-web server platform concept. mod_perl on the other hand is a complete implementation of the Apache API in Perl. This means that you get full access to every C API that Apache defines, and all the internals of Apache, if you want them. There are a lot of modules which build on this capability, one of these is Apache::Registry, which emulates a CGI-like environment in mod_perl, and provides huge performance boosts over normal CGI. But to get the full benefit of mod_perl it's often better to get right down into the Apache API and write server modules to do your work, just as is the case that if you need to improve a C based CGI, you will probably end up re-writing it as a server module. Which would be better to go with from a performance standpoint? I am assuming mod_perl because the instance does not need to be respawned everytime the interpretor is needed. Thanks! Yes, mod_perl is faster than CGI. There are a few "gotcha's", but these are well documented in the guide at http://perl.apache.org/guide/ -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: [performance/benchmark] printing techniques
On Wed, 7 Jun 2000, Stas Bekman wrote: Following Tim's comments here is the new benchmark. (I'll address the buffering issue in another post) use Benchmark; use Symbol; my $fh = gensym; open $fh, "/dev/null" or die; sub multi_print{ print $fh "!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML//EN\""; print $fh "HTML"; print $fh " HEAD"; print $fh "TITLE"; print $fh " Test page"; print $fh "/TITLE"; print $fh " /HEAD"; print $fh " BODY BGCOLOR=\"black\" TEXT=\"white\""; print $fh "H1 "; print $fh " Test page "; print $fh "/H1"; print $fh "A HREF=\"foo.html\"foo/A"; print $fh "HR"; print $fh " /BODY"; print $fh "/HTML"; } sub single_print{ print $fh qq{!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN" HTML HEAD TITLE Test page /TITLE /HEAD BODY BGCOLOR="black" TEXT="white" H1 Test page /H1 A HREF="foo.html"foo/A HR /BODY /HTML }; } sub here_print{ print $fh __EOT__; !DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN" HTML HEAD TITLE Test page /TITLE /HEAD BODY BGCOLOR="black" TEXT="white" H1 Test page /H1 A HREF="foo.html"foo/A HR /BODY /HTML __EOT__ } sub list_print{ print $fh "!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML//EN\"", "HTML", " HEAD", "TITLE", " Test page", "/TITLE", " /HEAD", " BODY BGCOLOR=\"black\" TEXT=\"white\"", "H1 ", " Test page ", "/H1", "A HREF=\"foo.html\"foo/A", "HR", " /BODY", "/HTML"; } timethese (500_000, { list_print = \list_print, multi_print = \multi_print, single_print = \single_print, here_print = \here_print, }); And the results are: single_print: 1 wallclock secs ( 1.74 usr + 0.05 sys = 1.79 CPU) here_print:3 wallclock secs ( 1.79 usr + 0.07 sys = 1.86 CPU) list_print:7 wallclock secs ( 6.57 usr + 0.01 sys = 6.58 CPU) multi_print: 10 wallclock secs (10.72 usr + 0.03 sys = 10.75 CPU) Numbers tell it all, I'single_print' is the fastest, 'here_print' is almost of the same speed, I'list_print' is quite slow and I'multi_print' is the slowest. If we run the same benchmark using the unbuffered prints by changing the beginning of the code to: use Symbol; my $fh = gensym; open $fh, "/dev/null" or die; # make all the calls unbuffered my $oldfh = select($fh); $| = 1; select($oldfh); And the results are: single_print: 4 wallclock secs ( 2.28 usr + 0.47 sys = 2.75 CPU) here_print:2 wallclock secs ( 2.45 usr + 0.45 sys = 2.90 CPU) list_print:7 wallclock secs ( 7.17 usr + 0.45 sys = 7.62 CPU) multi_print: 23 wallclock secs (17.52 usr + 5.72 sys = 23.24 CPU) The results are worse by the factor of 1.5 to 2, with only I'list_print' changed by very little. So if you want a better performance, you know what technique to use. I think this last line is misleading. The reality is that you're doing 500,000 iterations here. Even for the worst case scenario of multi_print with no buffering you're managing nearly 22,000 outputs a second. Now granted, the output isn't exactly of normal size, but I think what it comes down to is that the way you choose to print is going to make almost zero difference in any real world mod_perl application. The overhead of URL parsing, resource location, and actually running your handler is going to take far more overhead by the looks of things. Perhaps this section should be (re)moved into a posterity section, for it seems fairly un-informative to me. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: [performance/benchmark] printing techniques
On Wed, 7 Jun 2000, Eric Cholet wrote: So if you want a better performance, you know what technique to use. I think this last line is misleading. The reality is that you're doing 500,000 iterations here. Even for the worst case scenario of multi_print with no buffering you're managing nearly 22,000 outputs a second. Now granted, the output isn't exactly of normal size, but I think what it comes down to is that the way you choose to print is going to make almost zero difference in any real world mod_perl application. The overhead of URL parsing, resource location, and actually running your handler is going to take far more overhead by the looks of things. I don't understand what you're getting at. Does this mean that something shouldn't be optimized because there's something else in the process that is taking more time? For example I have a database powered site, the slowest part of request processing is fetching data from the database. Should I disregard any optimization not dealing with the database fetches ? These things add up, so don't you think that whatever can be optimized, should ? Of course the slowest stuff should be optimized first, but that doesn't mean that other optimisations are useless. Of course you can optimize forever, but some optimizations aren't going to make a whole lot of difference. This is one of those optimizations, judging by these benchmarks. Let Stas re-write this benchmark test as a handler() and see what kind of difference it makes. I'm willing to bet: barely any between averages. Perhaps I was a little strong: Lets not deprecate this part of the guide, just provide some realism in the conclusion. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: [performance/benchmark] printing techniques
On Wed, 7 Jun 2000, Eric Cholet wrote: This said, i hurry back to s/"constant strings"/'constant strings'/g; Those two are equal. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: Solution for: Re: $ENV{PATH} set by mod_perl script affectsmod_cgi scripts
On Tue, 6 Jun 2000, Ben Cohen wrote: Thanks to all for the helpful suggestions. Gunther Birzniek from the list suggested a solution that I've now tried and it works perfectly: {begin quote} I assume you are running with Apache:Registry? You could also save off the $ENV{PATH}... Go to the line that reads: eval { {$cv}($r, @_) } if $r-seqno; Ugh... Hate that syntax - can we patch it to: eval { $cv-($r, @_) } if $r-seqno; It's so much less cryptic. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: Warning about guide exceptions docs...
On Sun, 4 Jun 2000, Stas Bekman wrote: On Sat, 3 Jun 2000, Matt Sergeant wrote: Just a "heads up" about the exceptions section of the guide. Don't try and create more than one generic exception handler on your server. As I've just discovered it really confuses things. Create one class and one class only for handling exceptions globally. Matt, will you please send me an update? I still didn't get a chance to dig into your exceptions guide. OK - could you possibly email me the current perl.pod so that I don't have to go downloading the whole thing? Also, the die overloading needs a prototype to work like the real die(). Maybe I should put out a CPAN release: SimpleException.pm or something? Sounds good! Will you really do it? Well it would be trivial - I'll ask [EMAIL PROTECTED] -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: [new module] Apache::Dispatch
On Mon, 5 Jun 2000, Stas Bekman wrote: On Mon, 5 Jun 2000, Geoffrey Young wrote: hi all... I'm not sure if some you remember the idea Vivek and Matt had about creating a handler that mapped, say, http://localhost/Foo/doit to Foo-doit() anyway, the relevant part of the thread, including some code, can be seen here: http://marc.theaimsgroup.com/?l=apache-modperlm=95598609306936w=2 I was thinking of officially implementing the idea and wanted to get some design feedback first... My thoughts so far: * limit the response to content handling phase only (I'm not really sure of what utility other phases would be anyway) * limit the top-level qualifier for the module that can be executed, but give this control to the user. perhaps using PerlAddVar to allow only Apache::, Foo::, etc modules only is safe enough? Geoff, I think you will open a Pandora box by releasing this module. I don't see it'd give some real savings, but users will get hurt, badly. You shouldn't let the control into user hands! (I mean the clients!) There will be alway a module that you will not know about, or a function/method inside it you won't think about. It shouldn't be dangerous at all if you specify: PerlSetVar DispatchPrefix MyModule Then http://localhost/Foo/bar calls MyModule::Foo-bar() -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: Can't create custom configuration directives
I've snipped the lengthy explanation. Here's a few things you can check: There's no old version of your module in the ordinary perl lib directory, rather than the i386-foo/ directory. That you use DynaLoader. That you define $VERSION before the bootstrap line. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Warning about guide exceptions docs...
Just a "heads up" about the exceptions section of the guide. Don't try and create more than one generic exception handler on your server. As I've just discovered it really confuses things. Create one class and one class only for handling exceptions globally. Maybe I should put out a CPAN release: SimpleException.pm or something? -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Segfault in second time through sub request...
Running under httpd -X, first time through my subrequest goes fine, second time through it segfaults. Here's the non-debugging bt: Program received signal SIGSEGV, Segmentation fault. 0x8080757 in ap_copy_table () (gdb) bt #0 0x8080757 in ap_copy_table () #1 0x80937ef in ap_set_sub_req_protocol () #2 0x8098156 in ap_sub_req_lookup_file () #3 0x2af188 in XS_Apache_lookup_file () from /usr/lib/perl5/site_perl/5.005/i386-linux/auto/Apache/Apache.so #4 0x5f6b6f6f in ?? () Cannot access memory at address 0x62636f64 Probably not a huge amount of help, and it smacks of a bug in Apache, rather than mod_perl. Still a PITA for me ;-) Config: Apache 1.3.12, Perl 5.00503, mod_perl 1.24. Non-DSO. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: Segfault in second time through sub request...
Replying to my own post... I found the problem - I was caching a subrequest in the child's memory and trying to call lookup_* again with that cached request. Doesn't work for fairly obvious reasons. On Fri, 2 Jun 2000, Matt Sergeant wrote: Running under httpd -X, first time through my subrequest goes fine, second time through it segfaults. Here's the non-debugging bt: Program received signal SIGSEGV, Segmentation fault. 0x8080757 in ap_copy_table () (gdb) bt #0 0x8080757 in ap_copy_table () #1 0x80937ef in ap_set_sub_req_protocol () #2 0x8098156 in ap_sub_req_lookup_file () #3 0x2af188 in XS_Apache_lookup_file () from /usr/lib/perl5/site_perl/5.005/i386-linux/auto/Apache/Apache.so #4 0x5f6b6f6f in ?? () Cannot access memory at address 0x62636f64 Probably not a huge amount of help, and it smacks of a bug in Apache, rather than mod_perl. Still a PITA for me ;-) Config: Apache 1.3.12, Perl 5.00503, mod_perl 1.24. Non-DSO. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: jezndi (again)
On Fri, 2 Jun 2000, Paul wrote: I don't know why, but it seems a lot of the messages I send to this list are getting through, but also being tossed back at me with a nastygram from [EMAIL PROTECTED] accusing me of spamming. Easy enough for me to just block this address, but I thought I'd post one more notice in case someone responsible for that address gets this post and wants to check it to find out why my text-only messages from netscape/Yahoo!mail are being bounced. =o) Well they do contain spam/adverts in them... -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: [OT] Bean::*?
On Wed, 31 May 2000, DeWitt Clinton wrote: Hi all, Okay, this is a rather ridiculous question. I spent the weekend implementing a property based object model in Java. However, after I finished, someone in my company laughingly pointed out that I had just re-invented Java beans. While there were some minor advantages to the model I came up with, the overwhelmingly richer feature set of the java.beans.* framework made my efforts redundant at best. One of the advantages, however, is that the object model I was working on would translate very nicely to Perl. In fact, the language independent nature of this (other than OO being a requirement) was the real reason I bothered with it at all. My question is this -- has anyone written an implementation of the Java Bean standard in Perl? CORBA::ORBit? -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: JOB: Senior mod_perl/Unix engineer in San Diego
On Fri, 26 May 2000, Chris Thorman wrote: Compensation: This is a senior position at a rapidly growing company. Compensation will meet or exceed industry standards and will be in line with the skills and degree of self-direction and flexibility demonstrated by the successful candidate. Opportunities for growth and advancement will abound for the candidate that demonstrates the ability to do the work of 20 ordinary people while still leaving lots of time for a satisfying personal life. Aww dammnit, and I can only leap tall buildings in a single bound. Shucks. ;-) -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: High-volume mod_perl based ecommerce sites?
On Thu, 25 May 2000, Jason Bodnar wrote: Definitely read the perltoot (Tom's OO Tutorial). I've heard alot of good things about Damian Conway's OO Perl book but I haven't read it myself. Damian's book is the book to end all perl books in my opinion - I wouldn't dream of hiring anyone who hasn't read it yet. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: URL rewriting logging
On Fri, 26 May 2000, Robert wrote: Hi, I'm trying to use Eagle book-like URL rewriting to track sessions and I'd like to log session id in the 'user' field (so Analog can do all kind of usefull stats about sessions). I hoped I could just set 'user' throught Apache::Connection's user method, but it seems to be read-only. Is there any simple way to make Apache log some knowN string (say $ENV{SESSION_id}) to the user field? Or do I have to write complete LogHandler for it? Thanks for your help. $r-auth_name($username); -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
AxKit article
http://www.xml.com/pub/2000/05/24/axkit/index.html -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: [newbie]: Catching apache output.
On Thu, 25 May 2000, Alexei V. Alexandrov wrote: Hello modperl, The problem i would like to solve is that i want to catch the actual output from the apache. I have bought "Writing apache modules with perl and C" there are two examples on how to write thing like Echo or NavBar. These two scripts just parse the actual file that is requested by the client. The thing i want to make is that a module written in perl can catch the output (any script [php, cgi, etc.], any html file) and parse the output and make some changes to it (say like parsing href tags). Where should i look? The short and sweet of it is that you can't. Apache doesn't cache the output before sending it like some web servers (Roxen comes to mind), it sends it as you "print" so you can't. There are things you can do.. for example if your modules are all mod_perl modules you can use the techniques used by the Filter/Chain modules, or what AxKit does is overload Apache.pm's print() method and catches the output that way. Hope that helps... -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
RE: High-volume mod_perl based ecommerce sites?
On Thu, 25 May 2000, Jason Bodnar wrote: Could someone also give me a quick list (or a link to where I can find a list) of some high-profile sites that use Perl, and pull it off? http://www.slashdot.org Careful with this - it's a high traffic site, yes. But it doesn't exactly pull off the scaleable code base problem. It took ages for Pudge (Hi Chris!) to get it into a releaseable state, and even still it's not exactly a beautiful bit of code. The problem there again: Poor initial coder and zero design. Rob Malda wasn't even out of Uni when he started slashdot, and it's code followed it's evolution, turning into a big mess. I suspect there are probably days when Chris thinks about re-writing the whole thing from scratch ;-) -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: Cookies
On Tue, 23 May 2000, Jim Serio wrote: Like I said, the cookie is being set, but I can't read the cookie. Apache::Cookie-fetch('cookie_name'); doesn't work. this is a fixup handler? you shouldn't be sending the complete http header there. you should use $r-headers_out like you did in your original example. and, use telnet to see what your module is actually sending, or a log handler that dumps $r-as_string, or Apache::DumpHeaders (on cpan). This brings up a not-so-mod_perl question. Is there a way to telnet to name-based virtual hosts? Can I spoof the GET request? telnet address 80 GET / HTTP/1.1 Host: virtualhost return -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: global variables and reparsing question (low priority ;)
On Tue, 23 May 2000, Marc Lehmann wrote: On Tue, May 23, 2000 at 12:56:28AM -0500, Autarch [EMAIL PROTECTED] wrote: On Tue, 23 May 2000, Marc Lehmann wrote: stable (mod_perl really is very unstable for large applications). Apart Wow, I wish you'd warned me before I did several large applications using mod_perl. Fortunately, they haven't experienced any mod_perl related Well, add XML::Parser, some large and twisted perl modules (like XML::XSLT), do a lot of things dynamically (recompiling etc..) and apache segfaults quite a lot, at random times, and even with some weird effects (adding a comment makes httpd segfault, removing it and it works again). Fortunately, once you found a source form where httpd starts up, mod_perl seems to run fine for a long time. Hmm... AxKit does all this, and is very stable for me, and I've only had a couple of reports of segfaults, none of which went unsolved as far as I know... Ah well, there are still stability issues, but that's C programming for you ;-) -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: RFC: Apache::Request::Forms (or something similar)
On Tue, 23 May 2000, Peter Haworth wrote: brian moseley wrote: On Mon, 22 May 2000, Peter Haworth wrote: In light of the non-dependency on mod_perl, the Apache::Request::Form name is also out. I'd still rather not use the CGI::Form name, in case there are any current users whose interface would change, which is a shame, because that seems like the most appropriate name. Erk, any ideas, or proof that no-one uses the existing CGI::Form? HTML::Form? :) Well, duh! Why didn't I think of that? Unfortunately though, HTML::Form already exists, and doesn't really do the same kind of thing. How about HTML::FormGen? HTML::Forms? -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: not porn at all !
On Tue, 23 May 2000, Brigitte Defoort wrote: Hey... I hope that it is the correct address to post this : Josh is it ? ;-) It's not the correct address. You included a mod_perl mailing list, and your job has no perl content at all. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: mod_perl and IPC
On Mon, 22 May 2000, DeWitt Clinton wrote: The problem had to do with large numbers of objects in the cache. ... Right now, things are in a holding pattern because I'm finding a limit on the number of objects I can put in the cache (less than 100, so it is an issue). Hopefully Sam will offer some insight here. I ran into this with IPC::SharedCache a couple of months ago and had some discussion with Sam about it. The problem is a lack of shared memory segments and/or semaphores compiled into the kernel. IIRC, the default for both under Linux is 128. The BSD system I was trying to use only had 32 segments and something like 10 semaphore identifiers. In order to scale a system, you'll need to recompile the kernel with higher limits. - Matt
Re: Project Information
On Sun, 21 May 2000, (Allen Wilson) wrote: Good afternoon... My name is Allen Wilson and I am looking to join a mod-perl team. If there are any positions available on current projects, I would appreciate any information pertaining to the project and the available positions. Are you job hunting, or looking for a free software project to work on to hone your skills? -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: mod_perl and BSDi 4.1
On Fri, 19 May 2000, Russell Hay wrote: BSDi/4.1 ... cannot find libperl.so. Find the directory on your machine with libperl.so in it (probably /usr/libdata/perl5/i386-bsdos/5.00402/CORE or /usr/local/lib/perl5/5.00502/i386-bsdos/CORE/), add it to /etc/ld.so.conf, and run ldconfig. - Matt
Re: LARGE PERL footprint
On Fri, 19 May 2000, David Larkin wrote: I require a large array of ints in a real application, just stripped problem down to bear bones for demo. Is your array sparse by any chance? If not your modperl daemon is going to get _much_ larger after you populate that array. If it's sparse, consider using a hash - that will give you a slight speed penalty at the benefit of not consuming quite so much ram. Other things to consider: Re-write this critical bit of code in XS (not as hard as it sounds). Use a database. Use the filesystem. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: [preview] Search engine for the Guide
On Thu, 18 May 2000, Randy Kobes wrote: Another thing that was configured in is that words have to be at least 3 characters long, which seems reasonable, and also there's some stopwords that don't get indexed, as they're too common. This list of stopwords is built by hand - so far it only includes 'perl' and 'modperl'. Also, the maximum number of hits is set at 30. It should also index $/, etc. So limiting to 2char words is another broken aspect... But I'm not complaining! It's 100% better than it was. Maybe someone would like my code for a db backed search engine and fix that up to something that could work? It's all built in perl so you're free to add and remove stopwords or change the min word length as you like. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: RFC: Apache::Request::Forms (or something similar)
On Fri, 19 May 2000, Doug MacEachern wrote: personally, i like to have generic things written in c, things that won't change much or at all after they are first implemented (not including bug shaking). e.g. Apache::Request. both c and Perl are great languages and blend very well together. both have pros and cons, i try to use the right combo of both to balance these out. c is smaller. c is faster. Perl is much easier to code than c, so i like to save it for things that are difficult to code. generating html is not difficult. is it really worth the time to implement this in c? i'm not convinced yet either, but it is worth thinking about. we need to consider these things if we want to see the Apache/Perl combo improve as a development platform. think big picture. I would say that the bigger picture is definitely not generating HTML with functions - use templates or stylesheets. just my 2p. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
[ANNOUNCE] AxKit 0.65
This release of AxKit adds XSP support. XSP is a script embedded XML language that is language and output agnostic. All big words, translated to: You can write XSP pages in Perl, Java or Javascript (*). http://xml.sergeant.org/axkit/ Details on XSP: http://xml.apache.org/cocoon/xsp.html (*) Perl supported in AxKit, Java and Javascript supported in Cocoon. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: writing code that works on machines with or without mod_perl
On Thu, 18 May 2000, Kenneth Lee wrote: modperlers, does it make sense if i put some mod_perl specific codes inside an eval() so that the code runs on machines that have or haven't mod_perl installed? eval 'MOD_PERL_CODE' if $ENV{MOD_PERL}; use Apache (); my $r = Apache-request; ... MOD_PERL_CODE Better still: eval { die unless $ENV{MOD_PERL}; require Apache; my $r = $Apache-request; ... }; Then you've got no (at least much less than the above) run-time overhead. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: writing code that works on machines with or without mod_perl
On Thu, 18 May 2000, Kenneth Lee wrote: i know that, but it doesn't work if i use "use", since the block will be eval()'d at compile time: eval { die unless $ENV{MOD_PERL}; use Apache::Constants qw(:common); ... }; it complains if Apache::Constants is not installed. Since you didn't read the docs, here they are (relevant bit highlighted): % perldoc -f use =item use Module LIST =item use Module =item use Module VERSION LIST =item use VERSION Imports some semantics into the current package from the named module, generally by aliasing certain subroutine or variable names into your package. It is exactly equivalent to BEGIN { require Module; import Module LIST; } ^^^ except that Module Imust be a bareword. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: passing Apache::File to XS code that expects FILE *?
On Thu, 18 May 2000, Vivek Khera wrote: "DM" == Doug MacEachern [EMAIL PROTECTED] writes: DM On Wed, 17 May 2000, Matt Sergeant wrote: Well, this may be true, but if you load IO::File before startup then it's not too big a deal... DM but it still adds a great deal of bloat to the server. and it's oo DM interface, while very slick, adds quite a bit of runtime overhead, turn DM the sugar sour imo. In an embedded environment like mod_perl, then how do you suggest to deal with the dangling file handles problem? That is, I'm processing a file or two, and some error occurs. In a normal perl program, I'd exit or return out and then when the program terminates, it automagically closes all the files. In mod_perl, the auto-close doesn't happen until much later. With the OO interface, when the handle goes out of scope, such as a function call return, the file is implicitly closed. What other mechanism do you propose to handle this situation other than IO::File? I use it all the time myself. use Apache; use Fcntl qw/:DEFAULT :flock/; my $fh = Apache-gensym(); sysopen($fh, "file", O_RDONLY) || die "Can't open: $!"; flock($fh, LOCK_SH) || die "Can't lock: $!"; ... when $fh goes out of scope it's closed and unlocked. Also see the guide section on exception handling. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: [preview] Search engine for the Guide
On Fri, 19 May 2000, Stas Bekman wrote: Ok, We have a preview ready for you. Randy Kobes worked hard to prepare this one. So your comments are very welcome. If you like it we'll put this into production. Please keep either the list CC'ed or if you reply to me in person, make sure you keep Randy CC'ed -- all the kudos should go his way :) So: The search is at: http://theoryx5.uwinnipeg.ca/cgi-bin/guide-search and the split guide is at: http://theoryx5.uwinnipeg.ca/guide/ Looks cool, except can we take the guide splitting back 1 level? It seems to be split on =head2's, and should be split (IMO) on =head1's. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: [preview] Search engine for the Guide
On Fri, 19 May 2000, Stas Bekman wrote: Ok, We have a preview ready for you. Randy Kobes worked hard to prepare this one. So your comments are very welcome. If you like it we'll put this into production. Please keep either the list CC'ed or if you reply to me in person, make sure you keep Randy CC'ed -- all the kudos should go his way :) So: The search is at: http://theoryx5.uwinnipeg.ca/cgi-bin/guide-search and the split guide is at: http://theoryx5.uwinnipeg.ca/guide/ One more point... The indexer or the searcher (or both) has a broken tokenizer for anything involving perl. Try searching for Apache::Constants, for example. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: Apache::DBI and autocommit
On Tue, 16 May 2000, William Deegan wrote: Greetings, from the various perldocs and web pages I understand the following to be true. If autocommit is not set and a script exits the transaction will be rolled back. The question I have is when the database handle is re-used will the autocommit still be unset if the script that set it errors out? No. What you need to do is use an exception handler around your code (see the guide/perl.html) and commit if your code ran OK, or rollback if not. Here's how I do it (sort of - there's more code to it than this): eval { dispatch(); }; if ($@) { rollback; output('errorpage.template', $@-value); return OK; } return OK; However also ensure that your transaction handling is fine grained at the level its happening. I use DBIx::AnyDBD so that I can do this: $db-begin_tran; $db-do_stuff; # calls die() if it fails # non-db stuff that calls die() if it fails $db-commit; -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: Guide search engine (was Re: multiple copies of a module)
BTW: Your email client is broken and not wrapping words. On Wed, 17 May 2000, Jeremy Howard wrote: Stas Bekman wrote: Hold on, at this very moment a few mod_perl fellas are working on having a good search engine for the guide. Just give it some more time, I'm trying to bring the best so it'll take a while... I'm glad you brought this up again. Since I mentioned I'd be happy to host such a thing, and asked for suggestions, I've got a total of one (from Stas--thanks!). That suggestion was to use ht://dig http://www.htdig.org/. While htdig is a reasonable engine, Stas's idea is this needs to be "guide specific". Meaning what I'm not sure, but I'm assuming it means to pick out only certain words to index... Has anyone got a search engine up and running that they're happy with? I just wrote a very simple SQL based engine - so I would say I'm happy with that. It's fast and it's all in perl. I could very simply rip out the search parts of the code for someone to play with if they wanted to. Stas has made the good point that it needs to be able to hilight found words, since the pages are quite large. If anyone has a chance to do a bit of research about (free) search engines, I'd really appreciate it if you could let me know what you find out. It'd be nice publicity if it was mod_perl based, I guess, but it doesn't really matter. I think word highlighting is overrated. It's only necessary in this case because the guide is so damn huge now. The size problem could be eliminated by making the guide split itself up into smaller sections. My proposal would be to do that by converting the guide to docbookXML and use AxKit to display the resulting docbook pages. The AxKit docbook stylesheets are nice and friendly, and written in Perl, not some obscure XML stylesheet language. And after all that, it would make converting the guide to a format O'Reilly likes to publish (i.e. docbook), trivial. My only concern is that it seems a little odd to keep this just to the Guide. Wouldn't it be useful for the rest of perl.apache.org? I wouldn't have thought it's much extra work to add a drop-down box to search specific areas of the sight (the Guide being one)... perl.apache.org already has a search engine. If there's a good reason to have the Guide's search engine separate to the rest of perl.apache.org, should it have a separate domain (modperlguide.org?, guide.perl.apache.org?)? guide.modperl.org ? -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: Guide search engine (was Re: multiple copies of a module)
On Wed, 17 May 2000, Jeremy Howard wrote: I just wrote a very simple SQL based engine - so I would say I'm happy with that. It's fast and it's all in perl. I could very simply rip out the search parts of the code for someone to play with if they wanted to. Sounds good. Personally, I'd rather a simple engine we can fiddle with ourselves than a big system written in C. Does your engine generate a database from flat files? Is there some basic parameterisation (a 'stop list' for common words, definable 'keyword' characters, ...)? Well it's just perl, so there's a separate word tokenizer, a separate db inserter and a separate searcher (which is split into query parser and SQL builder). The db inserter is aware of "ignore words" which are stored in the DB. I think word highlighting is overrated. It's only necessary in this case because the guide is so damn huge now. The size problem could be eliminated by making the guide split itself up into smaller sections. My proposal would be to do that by converting the guide to docbookXML and use AxKit to display the resulting docbook pages. The AxKit docbook stylesheets are nice and friendly, and written in Perl, not some obscure XML stylesheet language. And after all that, it would make converting the guide to a format O'Reilly likes to publish (i.e. docbook), trivial. Your word highlighting statement is, I suspect, controversial. On the other hand, converting to docbook is unlikely to meet much resistance from users--as long as Stas doesn't mind maintaining it!... To get the best of both worlds, why not simply chain the search engine result through a filter that does the highlighting. I bet someone's written such a filter already--anyone? My only concern is that it seems a little odd to keep this just to the Guide. Wouldn't it be useful for the rest of perl.apache.org? I wouldn't have thought it's much extra work to add a drop-down box to search specific areas of the sight (the Guide being one)... perl.apache.org already has a search engine. So I've heard, but: * Where is it? (doing a Find on the front page doesn't show it) At the bottom of all guide pages. * Does it do highlighting? No. * Can you select a subset of the site? (e.g. just the Guide) No. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: Guide search engine (was Re: multiple copies of a module)
On Wed, 17 May 2000, Robin Berjon wrote: At 11:19 17/05/2000 -0500, Jeremy Howard wrote: Your word highlighting statement is, I suspect, controversial. On the other hand, converting to docbook is unlikely to meet much resistance from users--as long as Stas doesn't mind maintaining it!... To get the best of both worlds, why not simply chain the search engine result through a filter that does the highlighting. I bet someone's written such a filter already--anyone? I haven't played with it, but getting docbook out of the guide should be as easy as using Pod::DocBook. Fwiw, there's also been some work done on coming up with an xpod dtd, but I don't know how far it's advanced. I've played with Pod::DocBook, and it's a good start, but uses the DocBook SGML DTD, so you can't process it with XML tools. It also doesn't support =over =item =back, which is a pretty major limitation, IMHO. However patching it to support that shouldn't be too hard. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: passing Apache::File to XS code that expects FILE *?
On Wed, 17 May 2000, Jim Winstead wrote: Is there some trick to passing an Apache::File to a function from an XS module that expects a FILE *? There's too much perl magic going on in the Apache::File implementation for me to see where I can just pull the FILE * out. (Its not strictly necessary that I do this, of course, it would just be nice so I can use Apache::File-tmpfile(). Of course I can do the same basic thing with POSIX::tmpnam().) Or IO::File-new_tmpfile(); -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: passing Apache::File to XS code that expects FILE *?
On Wed, 17 May 2000, Jim Winstead wrote: On May 17, Matt Sergeant wrote: Or IO::File-new_tmpfile(); I'd rather not go there. http://marc.theaimsgroup.com/?l=apache-modperlm=95454378223412w=2 Well, this may be true, but if you load IO::File before startup then it's not too big a deal... Alternatively use File::Temp on CPAN, Apache-gensym(), and open()/sysopen(). -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
[ANNOUNCE] AxKit 0.62
Just to keep things ticking over, I've released 0.62 hot on the heels of 0.61. This is a minor update with a fix for the external parsed entities cache invalidation code (which now works!) and the addition of support for !--#include...-- for XPathScript (which allowed me to modularise the docbook stylesheets). AxKit: http://xml.sergeant.org/axkit/ Download: http://xml.sergeant.org/download/ -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: an idea for a PerlFixupHandler
On 14 May 2000, (Randal L. Schwartz) wrote: It always bugged me that for a PerlHandler, I needed to set *two* things... both the SetHandler and the PerlHandler. I have this idea that a good generic PerlFixupHandler would fix that. Instead of setting the mime-type to "text/html", we'd set the mime-type to "text/html;Apache::Registry" for those directories that wanted Registry mangling, or "text/html;HTML::Mason" for Mason, etc. That of course would skate through the standard MIME phase being unmatched, and the Fixup handler would see the semicolon, rip out the second part as a Perl handler, and unshift the handler while setting the handler to perl-script. So, I'd merely have to do things like AddHandler text/html;HTML::Mason .htm .html You mean AddType here I think... at the top level. And if I wanted a particular file not to be Mason, I could wrap it: Files col*.html ForceType text/html /Files Why not again just: Files col* AddType text/html .html /Files I bet mod_mime_magic could even then be used to reach inside a file and intuit the right PerlHandler based on content. And stacking would be easy: AddType text/html;Apache::OutputChain;Apache::SSIChain Any thoughts on this? Am I confused to think that MIME phase would not get in the way? Or should this go ahead of the core MIME so that it can do some magic redirection as well? For that, I think I'd like PerlTypeHandler ahead of the core MIME that can do stuff like this: PerlSetVar ReType 'text/(plain|html)=text/html;Apache::Registry' and then anything that made it to here that looks text/plain or text/html really gets mapped to Registry. Thoughts? Am I trying to be too generic? I like useful frameworks. :) Nah - go for it. AxKit's XMLFinder does something pretty similar, only in not quite as generic a way, so it would be perfectly possible. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
AxKit update
For those not on the axkit mailing list, I thought you might like a quickie AxKit update: First an introduction: AxKit is a standards based XML templating system, and much more. Ultimately it's an application server, but there's more work to do to reach that level, IMHO. Right now it can deliver transformed XML content to browsers, WAP devices and other media types, and it does so very quickly. AxKit 0.60 now has the following features: Transformation of XML using XSLT, using either Sablotron XSLT, or XML::XSLT. Transformation of XML using XPathScript, a powerful ASP-like XML scripting language of my own invention that is much less verbose than XSLT, and probably more powerful (but then I'm biased). Transformation using XMLNews::HTMLTemplate of either RDF or NITF news feeds. Transparent caching. "Cascading" of the output from one stylesheet system into another. A plugin system that allows you to build your own system for stylesheet choice, or media selection or other facilities. In progress is also a direct to output mode, using SAX, that allows output to start appearing as soon as the parser starts on your XML. This will be undeniably useful for database interaction. I'm also working on an XML compiler for this, that compiles XML into Perl code generating SAX events. If any of this gets your interest, then wander on over to http://xml.sergeant.org/axkit/ (xml.sergeant.org is 100% axkit built) -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: may be an offtopic question..
7On Wed, 10 May 2000, Niral Trivedi wrote: All, I am not sure whether this is the right place to ask this question... sorry if not.. My question is, Can we use custom database for authentication with Apache instead of default text based authentication??? Has anybody done that before?? I have looked apache site and cpan site.. but couldn't find any documentation... Can somebody give me some idea please?? I can do better than give you some advice! http://www.modperl.com/book/chapters/ch6.html -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: [Summation] 100% sessions?
On Wed, 10 May 2000, Jay Jacobs wrote: So as I see it there are essentially 2 *mostly* reliable ways, cookies and url-rewriting. Both have drawbacks and neither are 100%. There really isn't a way to cross-reference anything else (IP or login) becuase there are valid reasons for a user to come from multiple ip addresses during a session (albeit rare), and sessions may be needed without requiring a user to login. It also doesn't make sense to try to rely on both cookies and url-rewriting, that would just get sloppy and waste time. The only thing to do is to pick one or the other and deal with the drawbacks associated with that... URLS: - redirecting to a different site sends the session_id in the HTTP_REFERER in some browsers, which ruins it for the rest of the world ;) - requires site-wide url-rewriting or site-wide relative links (including things like "../../index.html" which seems ugly IMO) If you're doing site-wide URL re-writing, you might as well re-write outside URL's to a redirect CGI, so that the session doesn't go in the referer. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Wierd error log entry:
Apache::StatINC: Can't locate /usr/lib/perl5/site_perl/5.0 at /usr/lib/perl5/site_perl/5.005/i386-linux/Apache/StatINC.pm line 19. Granted this is a development server and I do some wierd stuff, but that's just bizarre... Any ideas? -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: Wierd error log entry:
On Wed, 10 May 2000, Ken Williams wrote: Perhaps it's the result of a line like "require 5.0;"? That's the only thing I can think of, I've never seen it before. Maybe... I wonder what would cause that to a) go in %INC, and b) go out of date... Ah well. Server restart got rid of the error :) -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: OT: Re: Most nonesense I've ever read about mod_perl
On Sun, 7 May 2000, Frank Mayhar wrote: Perl does have some good constructs for Web work, too. I've been writing a webstore and some stuff is really convenient that would be inconvenient in C. On the other hand, there's some stuff that I just wouldn't use Perl for, like, say, a system daemon, and there's other stuff that I _can't_ use it for, like in the kernel. Do a web search for perlfs - yes someone really did embed perl into the Linux kernel ;-) -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
RE: Most nonesense I've ever read about mod_perl
On Sun, 7 May 2000, Jeff Stuart wrote: [...rest of message deleted...] Every language has it use, the truly knowledgeable understand when to use each language:) Sam Amen to that!!! I think that this point and the point about writing GOOD algorithms are VERY important ones and I think that it's important that this be taught! I'm not sure if it's being taught now in school but in my day (GOD I sound old :)) (1987-1991) it wasn't. I hear this very much depends what Uni you go to. I'm always surprised to hear that people _don't_ just learn pure algorithms and techniques at Uni - that's certainly all that appears to be taught here in the UK (learning languages has to be done on your own time generally, even ones that are a core part of your course). -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
RE: [excitement :)] mod_perl rocks!
On Mon, 8 May 2000, Mark D Wolinski wrote: I don't mean to rain on the parade, but... The more important number would be to look at the number of unique IPs, which has only moderately grown. I installed mod_perl this weekend on my main server to start testing it, etc. Netcraft, of course, reports that I've got mod_perl, which may be reflected in the next updating of the mod_perl chart. But, since the number of unqiue IP's hasn't grown as much as the number of unique hosts, the chart just appears to say that some virtual host providers installed mod_perl on their servers. I think this is a bigger deal than perhaps you're making out... It doesn't matter to me how many single IP addresses get mod_perl - if the number of virtual hosts is going up with mod_perl on, that translates directly into more people having access to mod_perl, and more "servers" with mod_perl on. Whereas more IP addresses really doesn't tell you much - especially if an ISP hosts 200 hosts or more on 1 box! -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: Content negotiation headers
On Sun, 7 May 2000, Eric Jain wrote: How do I suppress content negotiation headers? Don't use mod_negotiation! I am using my own handler to convert XML to HTML on the fly: http://biodoc.ch/de/forum/2000/1/01.xml - HTTP/1.1 200 OK Date: Sun, 07 May 2000 20:40:52 GMT Server: Apache/1.3.9 (Unix) (SuSE/Linux) mod_perl/1.21 mod_ssl/2.4.7 OpenSSL/0.9.4 Connection: close Content-Type: text/html Perfect, except that I would like to hide the file ending. Enter mod_negotiotion: http://biodoc.ch/de/forum/2000/1/01 - HTTP/1.1 200 OK Date: Sun, 07 May 2000 20:40:21 GMT Server: Apache/1.3.9 (Unix) (SuSE/Linux) mod_perl/1.21 mod_ssl/2.4.7 OpenSSL/0.9.4 Content-Location: 01.xml Vary: negotiate TCN: choice Connection: close Content-Type: text/html This will obviously not be cached by any proxies :-( So how do I get rid of Content-Location, Vary and TCN? Install a handler that sets $r-filename: $r-filename($r-filename . '.xml') if -e ($r-filename . '.xml'); (note that AxKit can do all this for you, and comes with a FileSuffix style chooser which does similar to what you're after). -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Most nonesense I've ever read about mod_perl
http://slashdot.org/comments.pl?sid=00/05/05/0137201cid=250 There's a real good reply below, but terribly formatted (all italics). -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: Redirecting Problem --- Please look at this patiently.
On Fri, 5 May 2000, sadhanandham balaraman wrote: Hi Gurus, I'm facing a typical problem in Apache web server. The problem is that I want to call a perl script whenever a request is made to the server, and that script should be able to change the URI and submit to the server back. Make a PerlHandler that does $r-internal_redirect(URI); -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: PerlAddVar ?
On Fri, 5 May 2000, Stas Bekman wrote: On Wed, 3 May 2000, Doug MacEachern wrote: On Mon, 1 May 2000, Stas Bekman wrote: That [the name] would be confusing. How about: not if you think of it in terms of an apache table: PerlSetVar = ap_table_set PerlAddVar = ap_table_add OK, if this is the case, I'd prefer to have only one variable type -- an array - so $r-dir_config will always return a list. So if it's a sigular value, it'll be the first element. If you want it to be a hash -- you can have it as well. This makes thing simple and even eliminates the need for PerlSetVar, you can have only PerlAddVar for everything. And/or PerlSetVar to reset the list to 1 element. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
apxs and first class configuration
When building first class configuration directives, you run Makefile.PL and it says: which: no apxs in (/sbin:/usr/sbin:/bin:/usr/bin:/usr/X11R6/bin) apxs:Error: Sorry, no DSO support for Apache available apxs:Error: under your platform. Make sure the Apache apxs:Error: module mod_so is compiled into your server apxs:Error: binary `/usr/local/apache/bin/httpd'. But it compiles correctly anyhow. Is this just a check to make sure you have mod_so compiled in? -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Perl sections and custom configuration directives
I have someone on the AxKit list asking if there's a way to do configuration outside of .htaccess files. I figure rather than writing some new code to do this, Perl sections could be used. Will this work with custom directives, so basically could I do: Perl @AxAddStyleMap = ( [ 'text/xsl' = 'Apache::AxKit::Language::XSLT' ], ... ); /Perl I don't see why not, and I haven't tested it, but I just thought I'd double check here first. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: Perl sections and custom configuration directives
Answering myself: It works. Damn this product (mod_perl) is cool! On Fri, 5 May 2000, Matt Sergeant wrote: I have someone on the AxKit list asking if there's a way to do configuration outside of .htaccess files. I figure rather than writing some new code to do this, Perl sections could be used. Will this work with custom directives, so basically could I do: Perl @AxAddStyleMap = ( [ 'text/xsl' = 'Apache::AxKit::Language::XSLT' ], ... ); /Perl I don't see why not, and I haven't tested it, but I just thought I'd double check here first. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: Perl sections and custom configuration directives
On Fri, 5 May 2000, Stas Bekman wrote: On Fri, 5 May 2000, Matt Sergeant wrote: Answering myself: It works. Damn this product (mod_perl) is cool! On Fri, 5 May 2000, Matt Sergeant wrote: I have someone on the AxKit list asking if there's a way to do configuration outside of .htaccess files. I figure rather than writing some new code to do this, Perl sections could be used. Will this work with custom directives, so basically could I do: Perl @AxAddStyleMap = ( [ 'text/xsl' = 'Apache::AxKit::Language::XSLT' ], ... ); /Perl Even a cleaner solution, especially if you distribute some sample configuration files, would be to write this Perl file and load it with PerlModule AxConfig.pm: package Apache::ReadConfig; @AxAddStyleMap = ( [ 'text/xsl' = 'Apache::AxKit::Language::XSLT' ], ... ); 1; Actually the idea comes from setting up the style map based on an external XML site map, which would do things similar to apache's Files and Location directives, and then put them into the appropriate sections in Apache::ReadConfig. It's not something I'm going to ship with the product just now though - but someone wants the ability to do so. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: Perl sections and custom configuration directives
On Fri, 5 May 2000, brian moseley wrote: On Fri, 5 May 2000, Matt Sergeant wrote: Actually the idea comes from setting up the style map based on an external XML site map, which would do things similar to apache's Files and Location directives, and then put them into the appropriate sections in Apache::ReadConfig. It's not something I'm going to ship with the product just now though - but someone wants the ability to do so. hi, that's me :) actually matt, what i was shooting for was exposure to axkit's internal configuration data structures. i don't want to have to set up stuff in the $Apache::ReadConfig namespace and then have all the apache gears crank back around to your code. i'd rather have a configuration interface directly into your code. this way i can reconfigure axkit at runtime. using apache config i can't do that. Hmm... OK maybe a rethink is in order. Perhaps something like this: AxConfigReader MyModule And AxKit defaults to its internal module which reads the Ax* configuration directives directly? You could even inherit from the internal config reader for options you don't want to override. Sound OK? -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Memory leak in DBD::Connect()
Has anyone else come across a problem using DBD::Oracle, where connect() leaks approximately 80k every time it is used (which, in my case, is with every request child processes handle)? It's negligable with regular perl scripts, but I've seen child sizes grow to 40 megs before I killed them all. This problem is using the Oracle 8.1.5 libraries. Thanks, Matt
Re: PerlHandler stopped working???
On Fri, 5 May 2000, Doug MacEachern wrote: On Tue, 25 Apr 2000, Matt Sergeant wrote: I do now - just uploaded a new version. It's still not correct though - a proper fix would have to pull SetHandler out of mod_mime altogether, I guess. For example, say your config contains: oh yeah, i forgot, you can call mod_mime's fixup directly too: use Apache::Module (); my $rc = Apache::Module-top_module-find("mod_mime")-type_checker-($r); that digs the type_checker function pointer out of the mod_mime module structure and returns a reference to an anonymous xsub, which calls that function. neato, eh? Really really freaky. However I assume you saw the post - I solved it by doing push_handler() with a Fixup and returning DECLINED. That way mod_mime gets to do its stuff, and my content_type still gets set regardless. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: PerlAddVar ?
On Fri, 5 May 2000, Doug MacEachern wrote: On Thu, 4 May 2000, Matt Sergeant wrote: How do you get at $r in a directive handler? other way around, you get at directive handler config from a handler, which has been passed $r, e.g.: sub handler { my $r = shift; my $cfg = Apache::ModuleConfig-get($r, __PACKAGE__); $cfg would contain the data stashed by the directive handler. Ya - got that bit now. See AxKit 0.57 - it has an overloadable class for getting the configuration, so you can either get it from ModuleConfig() via .htaccess, or provide another method, such as brian wants to be able to emulate Cocoon 2's sitemap feature. This allows you to do that. Gotta love OO. For anyone interested, AxKit is proceeding at an incredible rate. We've now got an almost complete XSLT implementation curtesy of the Sablotron library, and lots of other excellent things going on. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: Apache::ModuleConfig
On Thu, 4 May 2000, raptor wrote: hi, someone to know is there Apache::ModuleConfig as separate package...? No, it comes with mod_perl. I think you require mod_perl 1.17 or higher. Can anyone on the mod_perl list confirm this? -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: search engine for the Guide (was Re: Why does $r-print()...)
On Thu, 4 May 2000, Stas Bekman wrote: On Wed, 3 May 2000, Matt Sergeant wrote: On Wed, 3 May 2000, Stas Bekman wrote: Yeah, I've been thinking about it. There was one site that has offered me to provide a good search engine and they did, but the problem is that they didn't keep up with new releases, so people were searching the outdated version, which is quite bad -- I've removed the reference to it, after asking them to update their copy for a few months, with no results. Can't we use WWW::Search - If I recall correctly some of the sites can be restricted to a domain, so you could build a search interface pretty easily. DESCRIPTION : This class is the parent for all access methods supported by the WWW::Search library. This library implements a Perl API to web-based search engines. It's not the search engine -- it's a Perl API to the search engines. We need a search engine not the API to it. Did I miss something? Yes. On some of the search engines (AltaVista springs to mind) you can search for things on particular web sites, or even links to particular web sites. So as long as AltaVista keeps its search contents up to date, you can leverage their engine. IIRC either Randall or Lincoln did a WebTechniques article about this a few months ago. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: Templates.
On Wed, 3 May 2000, Jason C. Leach wrote: hi, I'm looking for some good ideas on developing web sites w/ mod_perl. One think we were looking at was to write template HTML pages, and run them through a perl prg to replace home made tags w/ data. If what you're after is something to convert home made tags with data, then you're probably going to love AxKit. It's an XML based solution, using W3C standards, that offers a caching template solution for you. http://xml.sergeant.org/axkit/ (hopefully soon to be axkit.org, if Demon get moving on it...) -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: headless operation
On Tue, 2 May 2000, Tobias Hoellrich wrote: [EMAIL PROTECTED], for a future project I'm in the need to support two different ways how our web based service can be accessed: 1.) The traditional way: Handling user requests through a browser 2.) The "headless" way: Handling under-the-hood requests which basically perform the same service as in 1, but without ever generating valid HTML. Of course I want to write application logic only once and reuse it for both scenarios above. I looked at the various template/component systems (HTML::Mason, Embperl, HTML::Template) and get the impression that all of them are very much fall into category one, where they in some way or another expect to be driven from a browser. I checked into xmlrpc (http://www.xmlrpc.com) as a way to perform the under-the-hood operation, but it seems to become aweful when trying to integrate it with a mod_perl driven site using one of the template/component systems. What I'm basically planning to create is an application framework which does not tie into the HTML generation process, but is invoked from a driver that sends either browser-input or headless input to the application logic and then filters the output to generate either headless responses (XML probably) or HTML output. Has anybody done somthing like this before? Are there any pointers you mod_perl'ers want to share with me? I don't quite understand the "headless" concept you're referring to. It's not a language use I've come across before, so maybe further explanation is required. But before you get back to me, take a look at http://xml.sergeant.org/axkit/ and see if it's in any way what you're looking for. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: High memory usage [Was: Hi, goor morning modperlers]
First, please pick a sensible subject line... On Tue, 2 May 2000, FEITO Nazareno wrote: Hi, wassup ppl... I have a little problem, when I start up my apache 1.3.12 with mod_perl.1.23, it seem like he is eating all the memory, in a point that the computer almost doesn´t respond, I did try upping swap, I have 128MB ram and 512MB swap(is crazy for me, but I tryied), I don´t know what´s is the problem, my startservers are only 3 and minspare y maxspare are 3 and 6, I don´t have idea what i´m doing wrong, when the webserver is down the memory used is something like 70MB and swap used is 250MB but when the webserver is ^ Looks like you have a fairly serious problem without even using mod_perl! up, memory used is something like 127MB and swap 510MB, is amazing for me, and I can´t even have an idea of what´s going on... That's not a huge amount of help, but you reading the mod_perl guide will be: http://perl.apache.org/guide/ Look for the bits on memory usage (I'm sure someone here will give you a direct URL in about 5 minutes...). -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: headless operation
On Tue, 2 May 2000, Tim Gardner wrote: I am guessing that the "headless" version is what I do this when I am doing a shockwave piece which calls cgi scripts. I create an html version which provides a browser interface, albeit more boring, to the same basic code so that I can more easily test the cgi side. My strategy is to create two different versions of an "outputer" class, and within the code always call it to do the printing. For example $outputer-printResults( $x, $y); Then I create one outputer class which outputs proper html and one which outputs what the shockwave piece needs (much smaller). Then depending on parameters in the cgi calls or in the database, I instantiate the proper outputter to handle the call. This is exactly what AxKit is designed to achieve. It's not complete yet by any means. Join the list, start hacking and suggesting, and you'll shape where it goes from here. (with AxKit, association with different media devices from the standard W3C list is automatic, and picking a stylesheet based on alternates is just a plugin away). http://xml.sergeant.org/axkit/ -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: headless operation
On Tue, 2 May 2000, Autarch wrote: On Tue, 2 May 2000, Tobias Hoellrich wrote: for a future project I'm in the need to support two different ways how our web based service can be accessed: 1.) The traditional way: Handling user requests through a browser 2.) The "headless" way: Handling under-the-hood requests which basically perform the same service as in 1, but without ever generating valid HTML. I think that whay you want, contrary (and/or supplementary) to what some other people suggested, is to start by writing an API. This API should do everything _but_ handle the interface. This means it takes care of the database, business logic, report generation (in the form of data structures), etc. Then you write two interfaces against the API. One is an HTML based interface (which you could do with any of the wonderful perl embedding tools available. My personal preference is Mason). The other interface might be a set of command line executables (which can be run from cron) or an actually shell interface, or curses, or whatever. Right - this is how my *mumble* commercial project works, sort of. It's the old MVC methodology, if I'm not mistaken. This is actually how the whole XML approach of Cocoon and AxKit works, if you think about it. Your DTD is your API, your XML is one "record" or instance of that data, and the stylesheet(s) are your views on the object. At this time AxKit doesn't provide the kind of flexibility required to deliver the sort of dynamic application that serves from databases or user input using this kind of MVC system. But I'm working on it. Hopefully others will soon too... http://xml.sergeant.org/axkit/ -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
PerlAddVar ?
It would be nice, in my opinion, to have some way of doing: PerlAddVar Fred "Value 1" PerlAddVar Fred "Value 2" And then in your script: my @values = $r-dir_config('Fred'); which gets ("Value 1","Value 2") in @values. Any thoughts on this? (I'm not set on the name PerlAddVar, if that's anyone's concern - but what about the idea of it?) -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: PerlAddVar ?
On Mon, 1 May 2000, Stas Bekman wrote: It would be nice, in my opinion, to have some way of doing: PerlAddVar Fred "Value 1" PerlAddVar Fred "Value 2" And then in your script: my @values = $r-dir_config('Fred'); which gets ("Value 1","Value 2") in @values. Any thoughts on this? (I'm not set on the name PerlAddVar, if that's anyone's concern - but what about the idea of it?) That [the name] would be confusing. How about: PerlSetScalar Foo Bar PerlSetArray Foo A B C PerlPushArray Foo D PerlSetHash Foo key val Sounds cool to me. Naming things is still the hardest problem in Computer Science today... :-) -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: PerlAddVar ?
On Mon, 1 May 2000, Stas Bekman wrote: Would it not be possible / preferable to handle this kind of thing from within Perl blocks? (localized to location, directory etc. blocks, even) At least, something along these lines would cut down on the amount of configuration syntax that would needed to be coped with. Perl blocks get evaluated in the Apache::ReadConfig namespace, which knows nothing about location it's placed in. What you can do is to set FQDN variables from within Perl section: Perl $MyEnv::FOO = "bar" @MyEnv::FOO = qw(bar foo); %MyEnv::FOO{bar} = "barfoo"; /Perl It doesn't matter whether you define the package MyEnv or not, since it's autovivified when first time referenced to. and then in your code: print $MyEnv::FOO; will do. The thing I don't like about this is that I'm writing code for non-perl programmers. Littering .htaccess files with perl code is not something I want to be getting into. I don't mind too much the work around that is: PerlSetVar MyVar "name = value, \ other = value, \ etc = continued" but I don't particularly think its well suited to setting multiple values for non-programmers, and it breaks as soon as you start needing spaces in values or something wierd that happens to break the parser code (from the Eagle book). -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: PerlAddVar ?
On Mon, 1 May 2000, Richard Dice wrote: Perl blocks get evaluated in the Apache::ReadConfig namespace, which knows nothing about location it's placed in. What you can do is to set FQDN variables from within Perl section: Right... I didn't actually expect the perl block suggestion to be something that would work. I didn't know whether it would or not, actually. [Regardless, I wouldn't have known the reason why/why not, so your explanation was very helpful. :-) ] Still, I didn't feel out of place saying what I said because what was being discussed were the addition of new things to mod_perl, anyhow (e.g. PerlSetScalar, PerlSetArray, PerlPushArray, PerlSetHash). If we're talking about new stuff, then really, perhaps the way I suggested the new stuff could be included (via perl blocks, or local-perl, whatever, rather than a handful of new configuration directives) might be a good idea. For what its worth, I may just go ahead and start doing some XS code so that I can invent my own configuration directives. Its still an interesting discussion though - I wouldn't go as far as Stas, and add "PerlSetHash" (because they can easily be created from arrays), but PerlSetArray and PerlPushArray seem quite useful to me - although I don't think the name is quite right - if I'm doing stuff for non-programmers, the name "Array" means nothing to them, and "Push" means even less! ;-) -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: Strange mod_perl output problem - Followup
On Mon, 1 May 2000, Stas Bekman wrote: On Mon, 1 May 2000, Hughes, Ralph wrote: OK, After further investigation, it seems that Oraperl/DBI/DBD, etc. are not at fault! I'm formatting the data for display using Perl formats and displaying it as pre-formatted output using pre/pre tags. http://perl.apache.org/guide/porting.html#Using_format_and_write_ Ah so that's why I couldn't get it to work!!! I was trying to solve this yesterday, and was completely bamboozled why even the $^A and formline implementation didn't work (although I might have been doing something wrong). Anyway, the solution I came up with was just sprintf: ##.## becomes %2.2f .## becomes %4.2f Pad all strings with (" " x 80) before using, and set their length with: %.25s for a max 25 char string. Or prefix the string with (" " x 80) for right-justifying. Works like a charm. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: 2 server setup w/mod_proxy with a per-filename filter
On Mon, 1 May 2000, Martin A. Langhoff wrote: hi, I'm trying to implement a one light + one fat apache server setup and I'm .. wanting it to proxy everything that looks ~ \.pl$ See [EMAIL PROTECTED]">http://forum.swarthmore.edu/epigone/modperl/mimzhingleh/[EMAIL PROTECTED] for some mod_rewrite examples from a couple of weeks ago on this list. - Matt
[ANNOUNCE] AXDTK-0.05
The Apache XML Delivery Toolkit is now at version 0.05. This release automates the caching mechanism so that any stylesheet mechanism you develop yourself doesn't have to. It does this by carefully overloading the Apache object, and re-blessing $r into that new object, and also re-tieing STDOUT to that new package. Apache::XPathScript stylesheets are turning in at about 80 hits/sec on my development box (heavily overloaded box running Apache::StatINC and a lot of other stuff). http://xml.sergeant.org/axdtk/ -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: Apache::ASP rules [was: Sorry men]
On Fri, 28 Apr 2000, Francesc Guasch wrote: Ime Smits wrote: Bottomline: Apache::ASP made me really happy because now don't have to develop for IIS anymore and I can stick to my Linux environment. I can I'm also being told every once in a while by management about why we aren't using IIS, you know, when they receive marketing information. I have a list of lack of features in mind but I'll like to hear horror stories and so from somone who really used it. My boss has lots of lovely (NOT!) things to say about it. But then he is biased - he's the product manager for O'Reilly's WebSite. IIS does lots wrong, but it's not terrible. The API is a bit pants, and while it's very quick at static files, it cheats. See http://www.wgg.com/rants/benchmark.html for some details on that. It's also a bloated pig - and you thought Apache+mod_perl used up lots of RAM, wait till you get IIS+MTS+Index Server running -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
RE: Security in displaying arbitrary HTML
On Fri, 28 Apr 2000, Gerald Richter wrote: Gerald, what about Embperl, does it escape \x8b? No, there is no html escape for \x8b (and I guess the other one Matt mentioned is \0x8d for ) I know, so Embperl will not escape it, but this could be simply change by an entry in epchar.c. Any suggestion to what this should be escaped? Then I will make a patch. "#" . ord("\0x8a") . ";" Whatever that produces. Same for \0x8d. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: Security in displaying arbitrary HTML
On Fri, 28 Apr 2000, Marc Slemko wrote: On Thu, 27 Apr 2000, Matt Sergeant wrote: Unfortunately there's also a browser bug to contend with. They treat \x8b (I think that's the right code) as and there's a similar code for . Since most web developers are just doing s//lt;/g; they are open to attacks based on character sets like this. Sad, but true. Even our loved CGI.pm was (is?) open to this bug - I think Lincoln has fixed the HTMLEncode function now though. Mmm? Which browsers? Do they have to be configured for any particular character set? And can you provide an example that demonstrates it? I can't reproduce it... Well if you have Apache 1.3.12, it implicitly sets the Content-Encoding, or the character set, so this bug is minimised. But only on static content. If there's no character set in the Content-type or Content-Encoding the browser sniffer comes into play, and Netscape (IIRC) picks it up as Latin-1, or US-ASCII, I can't recall which. The details are all available over the web. Tom Christiansen had an excellent informative discussion about it on p5p - search the archives. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: Proxy front end behind 64k
On Fri, 28 Apr 2000, Dave Hodgkinson wrote: Matt Sergeant wrote: I'm behind a 64k leased line here (net access is *extremely* expensive here in the UK) and I was thinking, a proxy front end is probably really not necessary for me. Worst case scenario: I get 8 clients connecting to my at about 1KB/s - my pipe is maxed out anyway, so pushing them onto a proxy is just making more work! Just a thought for anyone thinking about a proxy front end - evaluate your pipe too. I couldn't get your CV: Not Found The requested URL /cgi-bin/xmerge.pl was not found on this server. Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request. That's OK - I'm snowed under work-wise for just about the rest of my life ;-) -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
[ANNOUNCE] AXDTK 0.01
The first release of the packaged up Apache XML Delivery Toolkit is now available for download, at http://xml.sergeant.org/download/ The kit bundles together the following modules: Apache::MimeXML Apache::XMLStylesheet Apache::NotXSLT Apache::XPathScript Theres not very much documentation available at the moment, after installing type "perldoc Apache::XMLStylesheet" to get general install instructions. Discussion of AXDTK takes place on the mailing list. Sent a mail to [EMAIL PROTECTED] to subscribe, or browse the archive at http://xml.sergeant.org/cgi-bin/ezmlm-cgi/2 -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Proxy front end behind 64k
I'm behind a 64k leased line here (net access is *extremely* expensive here in the UK) and I was thinking, a proxy front end is probably really not necessary for me. Worst case scenario: I get 8 clients connecting to my at about 1KB/s - my pipe is maxed out anyway, so pushing them onto a proxy is just making more work! Just a thought for anyone thinking about a proxy front end - evaluate your pipe too. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: Proxy front end behind 64k
On Thu, 27 Apr 2000, dreamwvr wrote: hi, so your saying that say 'squid' would not be productive? seems to me that if you are caching http and ftp stuff well that is going to provide you with the pseudo of more bandwidth.. since not all requests need to go beyond squid .. being delivered from the cache or chain of caches it is more efficient that's for sure.. and that always is better IMHO If I can't serve pages any faster, or to more people because of bandwidth limitations - what good can it do me? -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: Proxy front end behind 64k
On Thu, 27 Apr 2000 [EMAIL PROTECTED] wrote: You could however have someone with much more bandwidth than you use mod_proxy to proxy and cache your site. Like someone such as myself where bandwidth in the US is so cheap it's ridiculous. Upgrading to T1 size pipe in a couple weeks at $200/mo with DSL... hehe, too awesome. (384k now) So, lemme get this straight..., not only does UK have crazy webhosting laws, but you have to pay a ton for the bandwidth? Is the UK purposely trying to kill off any resemblance of the new economy in their country? Good lord! Yes. BT have decided they want all the money for themselves ;-) (it's about $500US/month for a 64k leased line - we're getting DSL in the cities this summer...) Okay... I'm really really off topic here. But... to answer your question, no mod_proxy would be a huge benefit strangely enough. Because your bandwidth is really small... your going to have clients grabbing data at 1KB/sec off a HUGE mod_perl process... 64 of of them to be not so exact :-) 8. This is 64Kb, not 64KB. I only wish it were 64KB - had to download NT4 Option pack yesterday... 1 mod_perl process could handle all the load you could possibly generate, and just let the mod_proxies build up and you'll see a lot lower memory usage on your box... seriously, in low bandwidth situations if your using the box for more than hosting (which I'd be willing to put good money on you are) then mod_proxy stands to give you tremendous benefits in the amount of free resources for other programs. I'm going to try it. I'll let people know... -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: Proxy front end behind 64k
On Thu, 27 Apr 2000, Matt Sergeant wrote: 1 mod_perl process could handle all the load you could possibly generate, and just let the mod_proxies build up and you'll see a lot lower memory usage on your box... seriously, in low bandwidth situations if your using the box for more than hosting (which I'd be willing to put good money on you are) then mod_proxy stands to give you tremendous benefits in the amount of free resources for other programs. I'm going to try it. I'll let people know... OK, I can't figure this out.. help me out here. I want to deal with my virtual hosts on the heavyweight server. The frontend server should just be a simple thing that I never have to touch. But when I do: ProxyPass / http://127.0.0.1:8080/ ProxyPassReverse / http://127.0.0.1:8080/ I get all requests going to the default virtual host, not the virtual host I'm requesting... -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
mod_proxy and Name based Virtual Hosts
OK, just to get this onto a different subject line... I can't seem to get mod_proxy to work on the front end with name based virtual hosts on the backend, I can only get it to work if I have name based virtual hosts on both ends. So I have a front end saying: NameVirtualHost 194.70.26.133 VirtualHost 194.70.26.133 ServerName xml.sergeant.org ProxyPass / http://xml.sergeant.org:8080/ ProxyPassReverse / http://xml.sergeant.org:8080/ /VirtualHost ... etc, for all nbvh's and almost the same on the backend: NameVirtualHost 194.70.26.133 VirtualHost 194.70.26.133 ServerName xml.sergeant.org DocumentRoot... ErrorLog... CustomLog... /VirtualHost ... etc, for all nbvh's. It can't be this hard, can it? I just want my vhosts config on 1 server - not both! And I don't want to be changing my setup - i.e. no mod_rewrite to rewrite to DocRoot/hostname - that's just annoying! -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: mod_proxy and Name based Virtual Hosts
On Thu, 27 Apr 2000, Drew Taylor wrote: I'm not very skilled in this area, but it looks like you are proxying to a backend on port 8080, but you never specify port 8080 in the config for the backend... Perhaps this is it? Or did you leave out part of the backend config? No - I just left that out - otherwise there would be a port conflict and one of the httpds wouldn't start. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: mod_proxy and Name based Virtual Hosts
On Thu, 27 Apr 2000, Vivek Khera wrote: "MS" == Matt Sergeant [EMAIL PROTECTED] writes: MS OK, just to get this onto a different subject line... I can't seem to get MS mod_proxy to work on the front end with name based virtual hosts on the MS backend, I can only get it to work if I have name based virtual hosts on MS both ends. So I have a front end saying: I personally use name-based on the front end and port-based on the backend. You have to do the virtualizing on the front end. Still doesn't solve the problem of editing only 1 config file, unless I generate a config file with perl :( -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: Security in displaying arbitrary HTML
On Thu, 27 Apr 2000, Vivek Khera wrote: "SC" == Steven Champeon [EMAIL PROTECTED] writes: SC developers and designers) for Webmonkey: SC http://hotwired.lycos.com/webmonkey/00/18/index3a.html SC If you want to see what sort of stuff the XSS problem opens you up for, SC just try appending ?tw=scriptalert("aha!");/script to the URL above. Why on earth would you take user input and output it verbatim to your pages? Rule number 1 of developing a web site is to never trust the user's input values. *Always* validate it against what you're expecting. Unfortunately there's also a browser bug to contend with. They treat \x8b (I think that's the right code) as and there's a similar code for . Since most web developers are just doing s//lt;/g; they are open to attacks based on character sets like this. Sad, but true. Even our loved CGI.pm was (is?) open to this bug - I think Lincoln has fixed the HTMLEncode function now though. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: $r-set_handlers behavior?
On Wed, 26 Apr 2000, Geoffrey Young wrote: good morning... I'm a bit confused about $r-set_handlers and $r-push_handlers behavior. Both are listed in the eagle book as being per-request methods, but man Apache lists them both as server configuration directives. I think, though, that I'm seeing set_handlers as persisting with the child and push_handlers as being per-request. Personally, I'd rather have set_handlers per-request as well for situations where I want to add a handler but reorder it with the existing ones as well... set_handlers is per-request. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Mailing list for AXDTK
I'm thinking of starting a separate mailing list for the Apache XML Delivery Toolkit. For those who don't know, the Apache XML Delivery Toolkit now offers the core functionality of Cocoon (http://xml.apache.org/cocoon/) in mod_perl, without the proprietary extension that cocoon has implemented (because everything they've done can be achieved without any ?cocoon? processing instructions). In the near future I hope to add the following: - XSLT parsing using Xalan (once it's ported to Linux and stable) - SQL Pre-processing - Embedded Perl (already available to some extent) If there's interest in this mail me direct and I'll set something up. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
AXDTK Mailing List open!
I had a pretty good response to the offer of a mailing list for the Apache XML Delivery Toolkit - and very fast too! So before more people say yes go ahead... send a blank mail to mailto:[EMAIL PROTECTED] and join the list! -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: [RFC] XML/Apache Templating with mod_perl
On Tue, 25 Apr 2000, Leslie Mikesell wrote: According to Matt Sergeant: In case you missed it - I just announce the Apache XML Delivery Toolkit to both the modperl list and the Perl-XML list. With it you can develop an XSLT Apache module in 13 lines of code (no caching, but it works). I saw it, but perhaps misinterpreted the 'not' in the xslt package. Is this intended to be fairly compatible with IIS's 'TransformNode' handling of xml/xsl (i.e. can I use the same xsl files)? No. It's definitely Not XSLT compatible. Sorry if it wasn't clear. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: PerlHandler stopped working???
On Tue, 25 Apr 2000, Doug MacEachern wrote: On Sat, 22 Apr 2000, Matt Sergeant wrote: On Sat, 22 Apr 2000, Matt Sergeant wrote: The only thing I can think of, is that Apache::MimeXML is somehow stopping the PerlHandler phase from being executed. Can it do that (but still allow the PerlFixupHandler phase to execute)??? OK, it was Apache::MimeXML... Which is very odd indeed. A bug in mod_perl by the looks of things. All I'm returning from Apache::MimeXML, btw, is OK or DECLINED. It was returning OK when PerlHandler stopped working. For now I'll disable it, and set Mime types manually for .xml files - but something is seriously not right there. as i already explained to matt in another email, if a TypeHandler returns OK, then mod_mime's type handler is not called. which means that SetHandler, AddType, etc., for that require will be ignored, unless you return DECLINED or implement them yourself (see Apache::MIME in ch8 of the eagle book) OK, I'm looking at that now, and I realise you're just passing it straight through to apache to do the right thing, but I think it's the wrong thing... Hear me out. Apache::MimeXML, like most non-PerlHandler handlers, is not designed as a replacement for mod_mime, but to build on it. If I detect something as XML, I'd like to say "OK, I'm done with this phase, let the next continue". Likewise if I don't want to handle this phase (i.e. I want mod_mime to do its thang) I return DECLINED. I don't want a MIME handler that's supposed to add to mod_mime to change the fact that I have a PerlHandler setup. In order to build Apache::MimeXML properly, I'd have to build an entire replacement for mod_mime, in XS, parsing the configuration for SetHandler, and I don't want to do that. I guess the problem is that mod_mime implements SetHandler - and I'm not convinced it should. If you were given the opportunity to do it all again I'd suggest it be done as follows: If a PerlTypeHandler returns OK, check if @{$r-get_handlers('PerlHandler')} is true (i.e. there's a PerlHandler waiting in the wings). If so, call $r-handler('perl-script'). If a PerlTypeHandler returns DECLINED, do it as you do now. That should be backwards compatible with most code, I think. And not interfere with things already setup in peoples httpd.conf/htaccess files. Alternatively have the OK case remain the same, and have a new return value (DONE?) that signfies to do the SetHandler part of mod_mime. Convince me I'm thinking wrong, then I can get back to some real work ;-) -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Apache XML Delivery Toolkit
I thought I'd just post here what I sent to the axdtk list, in case there are people here who don't know what the heck I'm talking about... If this perks your interest, send a mail to [EMAIL PROTECTED], download the toolkit, and start asking questions... What is the AXDTK? It's a system very much like Cocoon (http://xml.apache.org/cocoon/), but based on mod_perl. The idea is to allow a few things: - Stylesheets and XML files get associated automatically using the w3c semantics (http://www.w3.org/TR/xml-stylesheet). - Pre and Post processing can be performed using mod_perl stacked handlers. - Output is cached automatically if the underlying system uses DOM, otherwise the stylesheet processor developer has to implement caching himself, however the hooks are provided for this. - Stylesheets can cascade if based on DOM trees. For those unfamiliar with Cocoon, here's what happens: 1. A request comes in for an XML file. 2. If someone has installed a stacked handler prior to Apache::XMLStylesheet, that handler is called. One idea for such handlers is to extract a media type by browser sniffing (e.g. sets the media to "handheld"). Another idea is to take a preferred style from the querystring such as "style=no%20images". 3. Apache::XMLStylesheet checks its cache for a matching set of combined parameters (file, style, media). If the cache says that the XML file is older than the cache, use the list of stylesheets in the cache. If the cache is older than the file, parse the file for the xml-stylesheet processing instructions. 4. Check all the stylesheets exist. 5. If all the stylesheets exist, check the difference in timestamps between the xml file and all the stylesheet files, and the output cache file. If any are newer than the output cache file, then we re-run all the styles against the xml file. If they're all older we stop there, set the filename to the cached file, and return DECLINED. Apache obliges and returns the cached file to the browser for us. 6. If we re-create, we have mark off stylesheet type mappings against modules or functions. That is, the type="..." attribute in the stylesheet processing instruction gives us a pointer as to what module to use to process that stylesheet (based on some info in your httpd.conf file). We load that module and execute the function, passing in the stylesheet and the xml filename as parameters. 7. Modules based on the DOM can use the mod_perl API to set the $r-pnotes('dom_tree') value when they're done, rather than printing out the results. This is how the caching works, and how cascading works. Modules that aren't based on the DOM can't cascade... yet. 8. The resulting DOM tree is printed out both to the cache file and to the browser. The DOM tree is disposed of. That, in a nutshell, is it. Currently it's a work in progress. Today I hope to finish the version that does all the caching work and cascading stylesheets. If anyone wants to volunteer to help, here's what I consider needs done: - A cool web page ;-) - Get Xalan/Xerces working on Unix. This is pretty vital to the success, IMHO. See http://xml.apache.org/ - Write modules that plugin to this architecture that perform the following functionality: - SQL plugin using DBI. Should take an XML file (or DOM tree), parse out connection information and run the SQL. Produce another DOM tree and put that DOM tree in $r-pnotes('dom_tree') - Implement other stylesheet processors, like XPathScript and NotXSLT - Get the word out! Perl must not be behind in the XML world for much longer! Download AXDTK at http://xml.sergeant.org/download/ It's not well organised right now. The key component is Apache::XMLStylesheet. To use that you need XML::Parser and Apache::MimeXML(0.05). The only plugins available are Apache::XPathScript and Apache::XPath::NotXSLT, and an XSLT plugin that's in the list archive. -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org
Re: PerlHandler stopped working???
On Tue, 25 Apr 2000, Doug MacEachern wrote: I guess the problem is that mod_mime implements SetHandler - and I'm not convinced it should. If you were given the opportunity to do it all again understood, but this is how apache is designed, mod_perl is just going with the flow here. I'd suggest it be done as follows: If a PerlTypeHandler returns OK, check if @{$r-get_handlers('PerlHandler')} is true (i.e. there's a PerlHandler waiting in the wings). If so, call $r-handler('perl-script'). so why not just do that in your PerlTypeHandler? I do now - just uploaded a new version. It's still not correct though - a proper fix would have to pull SetHandler out of mod_mime altogether, I guess. For example, say your config contains: # PerlTypeHandler blah SetHandler perl-script PerlFixupHandler foo And in foo::handler() you call $r-push_handler('perl-script'). Suddenly it stops working when you uncomment the PerlTypeHandler there. I guess one important question is - why do we have to call SetHandler for PerlHandlers and not for any of the other handler phases. For all the other phases Apache/mod_perl automatically figures out if there's a handler installed and either runs perl code, or lets apache do its work. Why can't PerlHandler do the same? (as you can tell - I haven't dug into the internals of this - but I am curious). i don't think it's right for mod_perl to have that logic hardwired in. the other solution, is to let mod_mime do it's thang in the type phase, and do your thang in the fixup phase to override anything mod_mime did that you don't want. Right. So what's the point of PerlTypeHandler again?... ;-) Seriously though - Apache::MimeXML is the only PerlTypeHandler, as far as a quick search reveals, apart from your examples in the book. So maybe it would be worth investigating this further. A fixup handler is an interesting solution... but can I stack FixupHandlers? (I run most of my mod_perl code from a FixupHandler). Also, you didn't pick up on something I sent to the list in another thread... I have code that goes: if ($r-current_callback eq 'PerlFixupHandler') { $r-push_handlers('PerlHandler', \code); } else { code($r); } so that I can install the module as either a fixup or a PerlHandler. The fixup installed version is 4 times slower than calling the function directly - is that to be expected? (we're talking 20 requests/sec vs 80 here - a huge difference). -- Matt/ Fastnet Software Ltd. High Performance Web Specialists Providing mod_perl, XML, Sybase and Oracle solutions Email for training and consultancy availability. http://sergeant.org http://xml.sergeant.org