Re: RFC: Apache2::CloseKeepAlive
On Wed, Aug 26, 2009 at 6:43 AM, wrote: > I'm tentatively convinced it's worth contributing, but you're welcome > to tell me why it's not. I think that setting a very short Keep-Alive timeout would be almost as good. More importantly, you shouldn't serve any static files from mod_perl. You should serve those from a proxy instead. However, I don't see any problem with releasing your code. It's an interesting idea and shows a novel use of IPC::MMA. > • The plan is to use Module::Build and Apache::Test. > What else or something different? Those are fine. > • Is something as specialized as this worth including in the > GP Perl libraries? Or is it better to just ask where the > mod_perl2 library is? What's GP Perl? > In addition to where to put the module itself, it needs a place to > put a companion DB-building app and a place for the resulting DB. > These are not standard features of a module installation. > Probably I just ask the user where they should go? > If so, how do I use the answers? You can ask on the module authors list, but I think you'll find some examples of modules with executable scripts if you poke around CPAN a little. LWP comes with some. - Perrin
Re: RFC: Apache2::CloseKeepAlive
cr...@animalhead.com wrote: I'm working toward contributing a module with this working title to CPAN, and would like your comments and advice. A non-code comment, but there seems to be some consternation about the "same terms as perl itself" license. See the link below: http://perlbuzz.com/2009/07/help-end-licensing-under-same-terms-as-perl-itself.html Adam
Re: RFC: Apache2::CloseKeepAlive
On Aug 26, 2009, at 7:03 AM, Perrin Harkins wrote: What's GP Perl? The general purpose Perl library is headed at /usr/local/lib/perl5 on my system. What I was asking is whether there's any reason to put Apache2::CloseKeepAlive in it, being as it's so specialized, or just copy it to a mod_perl2-specific library (/usr/local/apache2/plib on my system). Maybe the latter if the user says such a thing exists, else the former? cmac
Re: RFC: Apache2::CloseKeepAlive
On Wed, Aug 26, 2009 at 11:41 AM, wrote: > The general purpose Perl library is headed at /usr/local/lib/perl5 > on my system. What I was asking is whether there's any reason > to put Apache2::CloseKeepAlive in it, being as it's so specialized, > or just copy it to a mod_perl2-specific library (/usr/local/apache2/plib > on my system). Maybe the latter if the user says such a thing exists, > else the former? You don't get to choose that. You just do a standard Module::Build distribution, and if people want to install your module somewhere outside of the site_lib, they specify that at install time. - Perrin
Re: RFC: Apache2::CloseKeepAlive
On Aug 26, 2009, at 7:39 AM, Adam Prime wrote: A non-code comment, but there seems to be some consternation about the "same terms as perl itself" license. See the link below: http://perlbuzz.com/2009/07/help-end-licensing-under-same-terms-as- perl-itself.html Inside many programmers, nascent lawyers lurk? The linked-to CPAN Licensing Guidelines http://www.perlfoundation.org/cpan_licensing_guidelines was easier to read and didn't cut off the right sides of code segments. I modified the end of http://www.animalhead.com/CloseKeepAlive.html accordingly. Thanks, cmac
Re: RFC: Apache2::CloseKeepAlive
On Aug 26, 2009, at 9:05 AM, Perrin Harkins wrote: On Wed, Aug 26, 2009 at 11:41 AM, wrote: The general purpose Perl library is headed at /usr/local/lib/perl5 on my system. What I was asking is whether there's any reason to put Apache2::CloseKeepAlive in it, being as it's so specialized, or just copy it to a mod_perl2-specific library (/usr/local/ apache2/plib on my system). Maybe the latter if the user says such a thing exists, else the former? You don't get to choose that. You just do a standard Module::Build distribution, and if people want to install your module somewhere outside of the site_lib, they specify that at install time. - Perrin How does one specify that? It sounds like esoteric knowledge that only a few users will know, and I'd like to empower people not to clog up their GP Perl library with a module that can be used in exactly one place.
Re: RFC: Apache2::CloseKeepAlive
On Wed, Aug 26, 2009 at 12:21 PM, wrote: > How does one specify that? It sounds like esoteric knowledge that > only a few users will know, and I'd like to empower people not to > clog up their GP Perl library with a module that can be used in > exactly one place. It's in the docs for Module::Build. Seriously, you have no control over where users install your module. Attempting to control it is against the principles of CPAN distributions. - Perrin
Re: RFC: Apache2::CloseKeepAlive
Including a note about how to do something in an installation dialog, for people who wouldn't otherwise know, is not controlling anything. If that's against CPAN principles, the principles need some work! cmac On Aug 26, 2009, at 9:30 AM, Perrin Harkins wrote: On Wed, Aug 26, 2009 at 12:21 PM, wrote: How does one specify that? It sounds like esoteric knowledge that only a few users will know, and I'd like to empower people not to clog up their GP Perl library with a module that can be used in exactly one place. It's in the docs for Module::Build. Seriously, you have no control over where users install your module. Attempting to control it is against the principles of CPAN distributions. - Perrin
Re: RFC: Apache2::CloseKeepAlive
Seriously, this is not something you need to be concerned about. There are dozens of places for people to get information about how to install CPAN modules. Most people use the CPAN shell and set it up to put things where they want them. - Perrin On Wed, Aug 26, 2009 at 1:31 PM, wrote: > Including a note about how to do something in an installation dialog, > for people who wouldn't otherwise know, is not controlling anything. > If that's against CPAN principles, the principles need some work! > > cmac > > On Aug 26, 2009, at 9:30 AM, Perrin Harkins wrote: > >> On Wed, Aug 26, 2009 at 12:21 PM, wrote: >>> >>> How does one specify that? It sounds like esoteric knowledge that >>> only a few users will know, and I'd like to empower people not to >>> clog up their GP Perl library with a module that can be used in >>> exactly one place. >> >> It's in the docs for Module::Build. Seriously, you have no control >> over where users install your module. Attempting to control it is >> against the principles of CPAN distributions. >> >> - Perrin > >
Re: RFC: Apache2::CloseKeepAlive
cr...@animalhead.com wrote: Including a note about how to do something in an installation dialog, for people who wouldn't otherwise know, is not controlling anything. Please don't put a dialog in your installation process! CPAN is supposed to be automatic after it's been configured. There are some rare occasions where it makes sense to ask some questions, but where you want the module installed is not one of them. CPAN takes care of that. -- Michael Peters Plus Three, LP
Re: RFC: Apache2::CloseKeepAlive
The installation process needs to know at least: 1. where the accompanying DB-building script should go 2. where the DB that the app builds should go How can such necessary things be determined, other than by asking the user? I was just going to add a note before or after these two queries, that if they have a mod_perl2 library and want the module to go there, they should "do this". As I just wrote to Perrin off-list, since only a tiny fraction of CPAN downloads are mod_perl2 modules, no one can configure CPAN for a mod_perl2 library. Perhaps my attitude toward installation dialogs has been warped by recent experience with my previous CPAN module IPC::MMA. The install for version 0.54 contained no prompts. Its results came out 75% unknown, because few test environments included a necessary C library 'mm'. One of the chief testers suggested I should add Devel::CheckLib to Makefile.PL. So now if Makefile.PL doesn't find 'mm' in /usr/local/lib, it prompts the user to enter the path to 'mm', which is then passed to Devel::CheckLib. The query is accompanied by a note showing the URL from which 'mm' can be downloaded. On the other hand, my latest version 0.58 has only been tested by one tester, a week after uploading it. Maybe that's because of the new query in Makefile.PL! Best Regards, cmac On Aug 26, 2009, at 10:44 AM, Michael Peters wrote: cr...@animalhead.com wrote: Including a note about how to do something in an installation dialog, for people who wouldn't otherwise know, is not controlling anything. Please don't put a dialog in your installation process! CPAN is supposed to be automatic after it's been configured. There are some rare occasions where it makes sense to ask some questions, but where you want the module installed is not one of them. CPAN takes care of that. -- Michael Peters Plus Three, LP
Re: RFC: Apache2::CloseKeepAlive
cr...@animalhead.com wrote: The installation process needs to know at least: 1. where the accompanying DB-building script should go Module::Build knows where to put bin/ or script/ files and it does so according to how Perl and CPAN are configured. I assume EU::MM does too. 2. where the DB that the app builds should go That's probably something your application should allow to be specified, either a command line option to the script or an option to the module, or both. How can such necessary things be determined, other than by asking the user? If it's a run time option (and can be changed or multiple programs can use this and thus want different values) don't ask for it at install time. I was just going to add a note before or after these two queries, that if they have a mod_perl2 library and want the module to go there, they should "do this". Putting extra docs in your module is ok, but I personally wouldn't repeat the docs that are already out there. As I just wrote to Perrin off-list, since only a tiny fraction of CPAN downloads are mod_perl2 modules, no one can configure CPAN for a mod_perl2 library. All perl libraries should go in the same place IMO. Whether it's Apache specific, or GTK or something else. I don't want to have to mess with @INC just because someone put something someplace non-standard. So now if Makefile.PL doesn't find 'mm' in /usr/local/lib, it prompts the user to enter the path to 'mm', which is then passed to Devel::CheckLib. The query is accompanied by a note showing the URL from which 'mm' can be downloaded. Personally, I'd just bomb out if you can't find the lib (and Devel::CheckLib is good for that). Give the user a message that you can't find the library and then exit. I doubt that someone who has a library that's not in the system libs will know where it is, and if they do they can just as easily set LD_LIBRARY_PATH themselves. On the other hand, my latest version 0.58 has only been tested by one tester, a week after uploading it. Maybe that's because of the new query in Makefile.PL! That's why prompts are evil. Automated tests won't work. So just bomb out and at least you'll get lots of UNKNOWN test results which is better than nothing :) -- Michael Peters Plus Three, LP
Re: RFC: Apache2::CloseKeepAlive
On Aug 26, 2009, at 11:44 AM, Michael Peters wrote: cr...@animalhead.com wrote: The installation process needs to know at least: 1. where the accompanying DB-building script should go Module::Build knows where to put bin/ or script/ files and it does so according to how Perl and CPAN are configured. I assume EU::MM does too. So if I put the script in a subdirectory bin of Apache2- CloseKeepAlive-n.nn/, the install step should automatically put it somewhere in the path. This is good information, thank you! I have no idea what EU::MM is. Candy from Europe? 2. where the DB that the app builds should go That's probably something your application should allow to be specified, either a command line option to the script or an option to the module, or both. My plan was to run the script the first time under Build.PL, to verify that it succeeds and because without the DB there can be no testing. In which case, asking the user where to put the DB, and propagating the answer to the script source and the module source, seems to me like the easiest course for everyone. This kind of prompt is not evil, in that the automated testers can be handled by giving the query a default, maybe '/tmp'. How can such necessary things be determined, other than by asking the user? If it's a run time option (and can be changed or multiple programs can use this and thus want different values) don't ask for it at install time. It's not a run time option. The subject is a mod_perl2 script, that can only be used within the Apache2 server. I can't conceive why the DB name would ever change, and multiple programs won't use the module. I was just going to add a note before or after these two queries, that if they have a mod_perl2 library and want the module to go there, they should "do this". Putting extra docs in your module is ok, but I personally wouldn't repeat the docs that are already out there. I personally regard "docs already out there" as things that should be restated "in here" whenever the user can profit from them. As I just wrote to Perrin off-list, since only a tiny fraction of CPAN downloads are mod_perl2 modules, no one can configure CPAN for a mod_perl2 library. All perl libraries should go in the same place IMO. Whether it's Apache specific, or GTK or something else. I don't want to have to mess with @INC just because someone put something someplace non- standard. It seems that you and Perrin are in the same boat, which has "One Perl Library For All" painted on it. I am in a different boat with the authors of various mod_perl books. I set up my site under the guidance of "Practical mod_perl" p. 34: "Now you have to select a directory where all the mod_perl scripts and modules will be placed. We usually create a directory called modperl under our home directory for this purpose (e.g., /home/stas/modperl), but it is also common to create a directory called perl under your Apache server root, such as /usr/local/apache/perl." (It's /usr/local/apache2/plib on my site.) This theme is echoed on p. 58 of the "mod_perl2 User's Guide": "You often need to adjust the contents of @INC before the server starts, usually to give the server additional places to look for your mod_perl code. There are several ways to do this: • startup.pl In the startup file you can use the lib pragma like so: use lib qw(/home/httpd/projectl/lib/tmp/lib); use lib qw(/home/httpd/project2/lib); • httpd.conf In httpd.conf you can use the PerlSwitches directive to pass arguments to Perl as you do from the command line. PerlSwitches -I/home/httpd/projectl/lib -I/tmp/lib PerlSwitches -I/home/httpd/project2/lib" And on p. 61 of the "mod_perl Developer's Cookbook": "For a similar option, configure your startup.pl as use lib qw(/home/www/lib); ... By default, mod_perl adds two directories to @INC: ServerRoot/ and ServerRoot/lib/perl/, where ServerRoot is the value of the ServerRoot directive. ... If you are using a startup.pl to preload your modules, you will probably need to take advantage of the use lib syntax." Given all this, I blithely talk about "the mod_perl library" as if it's a standard part of every mod_perl site, and you and Perrin regard me as someone from another planet. Which is actually true, but that's another story. So now if Makefile.PL doesn't find 'mm' in /usr/local/lib, it prompts the user to enter the path to 'mm', which is then passed to Devel::CheckLib. The query is accompanied by a note showing the URL from which 'mm' can be downloaded. Personally, I'd just bomb out if you can't find the lib (and Devel::CheckLib is good for that). Give the user a message that you can't find the library and then exit. I doubt that someone who has a library that's not in the system libs will know where it is, and if they do they can just as easily set LD_LIBRARY_PATH themselves. This happens to me fairly reg
Re: RFC: Apache2::CloseKeepAlive
On Wed, Aug 26, 2009 at 9:23 PM, wrote: > I have no idea what EU::MM is. Candy from Europe? Good guess, but it's actually ExtUtils::MakeMaker. It's the CPAN install mechanism that Module::Build was written to replace. > My plan was to run the script the first time under Build.PL, to > verify that it succeeds and because without the DB there can be no > testing. I'd suggest trying to find another way. Installers that ask questions are a real problem for automated scripts and distribution packagers and are definitely discouraged. Take a look at the module authors list for more discussion on this. There are some standard ways of allowing people to skip prompts. One thing you could do for testing is to ship a small set of test files and a pre-built db for them. > It's not a run time option. The subject is a mod_perl2 script, that > can only be used within the Apache2 server. I can't conceive why the > DB name would ever change, and multiple programs won't use the module. Some people have many apache installations on one machine, or may install new ones after installing your module. > I personally regard "docs already out there" as things that should > be restated "in here" whenever the user can profit from them. There's no harm in giving them a pointer to the relevant part of the Module::Build docs or an example of using the correct flags. Go ahead. > It seems that you and Perrin are in the same boat, which has "One Perl > Library For All" painted on it. I am in a different boat with the > authors of various mod_perl books. I set up my site under the guidance > of "Practical mod_perl" p. 34: Sorry, I think you're mis-reading the advice in these books. What the authors are talking about there is your own local code, not CPAN modules. CPAN modules have a standard location and a standard mechanism for changing it for special cases. On the other hand, you would never put your own local code in the site_perl directory, so that goes in your mod_perl lib directory and gets added to @INC. I hope that explains why we're surprised to hear this idea of putting mod_perl modules in a separate place -- it's one I've never heard before. - Perrin
Re: RFC: Apache2::CloseKeepAlive
Perrin Harkins wrote: It's not a run time option. The subject is a mod_perl2 script, that can only be used within the Apache2 server. I can't conceive why the DB name would ever change, and multiple programs won't use the module. Some people have many apache installations on one machine, or may install new ones after installing your module. This is the main point I guess I didn't clearly express. Don't assume that just because you have only 1 apache installed that everyone else does too. Also, if you deviate from what the standard install tools do (which is install things based on how CPAN is configured on that machine) then actually make it harder for people who need to install your module in a non-standard place. This is pretty common for dev environments. I (and most Perl people I would assume) already know how to tell EU::MM or Module::Build how to put things where I want them. If you do it differently it makes our lives harder. I guess what it comes down to is that putting modules into non-standard places is pretty common, so please don't implement your own way of doing that. -- Michael Peters Plus Three, LP
Re: RFC: Apache2::CloseKeepAlive
On Wed, Aug 26, 2009 at 3:43 AM, wrote: > > > The short story is that it's a mod_perl2 mechanism for optimizing > the KeepAlive option in the Apache2 server. The long story is at > http://www.animalhead.com/CloseKeepAlive.html Interesting. But, is there really a probem to solve here? Is the point here to free up mod_perl processes faster so that they can handle other connections? Does that really have anything to do with better performace in serving up pages? I don't think you can look at file extensions and determine when to close the connection. Can you count on how the browser will send requests? For example, javascript probably should be loaded last in the HTML file. Then the bulk of images are often loaded via css. Are you scanning the css, too? What if you have multiple servers? Connections will be on different machines. If customizing the keep alive is that critical then I think it's time to look at other techniques. Use a balancer and let it decided how to manage the connections (perlbal keeps the connection open forever). Use sprite sheets to reduce the number of required connections. Use a CDN and cache static content forever -- and rarely hit your serve. In the end, each "page view" should only be one request to your mod_perl server anyway. -- Bill Moseley mose...@hank.org
Re: RFC: Apache2::CloseKeepAlive
On Aug 27, 2009, at 10:43 AM, Bill Moseley wrote: On Wed, Aug 26, 2009 at 3:43 AM, wrote: The short story is that it's a mod_perl2 mechanism for optimizing the KeepAlive option in the Apache2 server. The long story is at http://www.animalhead.com/CloseKeepAlive.html Interesting. But, is there really a probem to solve here? Is the point here to free up mod_perl processes faster so that they can handle other connections? Does that really have anything to do with better performace in serving up pages? Yes that's the point, but I would substitute 'apache processes' for 'mod_perl processes'. My site's front page preloads sixty-some jpg thumbnails. Years ago it was evident that KeepAlive On was needed to allow this to complete in a reasonable length of time. e.g., on a 1.5M ADSL line it takes 2-3 seconds with KA on, 6-8 seconds with KA off. So it seems that connection establishment and process dispatch take a while. Even though the Javascript that does the preloading on my front page only requests 3 images ahead of download completion, the response shown to a quick user clicking on a link is noticeably degraded by the preload stream. So you want to get the preloading over with before most users will click on a link. I decided to write this module about a year ago, when I read an article that the latest Firefoxes would open up to 6 connections to download supporting files, and that the MSIE team was considering leapfrogging them to 8. With my cheapskate ISP plan limiting the total number of processes, hanging 6 or 8 of them at a time (with a plain KeepAlive On) was a pretty scary idea. I don't think you can look at file extensions and determine when to close the connection. Can you count on how the browser will send requests? Maybe not strictly, but in general browsers request files in the order they are requested in the .html file. In the case of a lot of .jpg's the browser may take their relative file sizes into account. For example, javascript probably should be loaded last in the HTML file. Javascript that is involved in building the pages wants to get called out as early in the header as possible, either before or after the css. JavaScript that only interacts with the user could come last, but by doing that you take the chance of responding badly to a "quick" user. Then the bulk of images are often loaded via css. Are you scanning the css, too? The scanning script that accompanies the module could easily do this. I will count your email as a first request for that feature, and if I get 2 or 3 I'll add it. If that happens, can I count on you for a sample of a .css that loads images? What if you have multiple servers? Connections will be on different machines. Clearly that's a different environment than CloseKeepAlive is designed for. If customizing the keep alive is that critical then I think it's time to look at other techniques. Use a balancer and let it decided how to manage the connections (perlbal keeps the connection open forever). Anything that keeps a connection open forever is going to end up with a lot of connections? Use sprite sheets to reduce the number of required connections. I looked up "sprite sheets" and they seem to be graphics that include lots of little tiled figures used in video games. How does a browser know to split them up into separate images that pages can then call out individually? Use a CDN and cache static content forever -- and rarely hit your serve. In the end, each "page view" should only be one request to your mod_perl server anyway. My site uses all defined headers to encourage proxies and browsers to cache files. The performance of a site is the cross-product of all of the things that it does to improve performance. But you can't just look at it from the server's point of view. CKA is motivated by a desire to give the best possible performance/ responsiveness at the browser, that's consistent with keeping the load on the server reasonable.
Re: RFC: Apache2::CloseKeepAlive
On Thu, Aug 27, 2009 at 5:20 PM, wrote: > > Yes that's the point, but I would substitute 'apache processes' for > 'mod_perl processes'. > This is a mod_perl module, right? My site's front page preloads sixty-some jpg thumbnails. Years ago it was > evident that KeepAlive On was needed to allow this to complete in a > reasonable length of time. e.g., on a 1.5M ADSL line it takes 2-3 seconds > with KA on, 6-8 seconds with KA off. So it seems that connection > establishment and process dispatch take a while. > Got YSlow installed? Run it on your site. Have you reviewed this? http://developer.yahoo.com/performance/rules.html That's where I'd start. Others: http://www.websiteoptimization.com/services/analyze/ http://code.google.com/speed/page-speed/ http://stevesouders.com/hpws/rules.php > Anything that keeps a connection open forever is going to end up with a lot > of connections? > They are pooled. http://www.danga.com/perlbal/ > I looked up "sprite sheets" and they seem to be graphics that include lots > of little tiled figures used in video games. > How does a browser know to split them up into separate images that pages > can then call out individually? > Ask Google ;) http://www.websiteoptimization.com/speed/tweak/css-sprites/ -- Bill Moseley mose...@hank.org
Re: RFC: Apache2::CloseKeepAlive
cr...@animalhead.com schrieb: On Aug 27, 2009, at 10:43 AM, Bill Moseley wrote: Use sprite sheets to reduce the number of required connections. I looked up "sprite sheets" and they seem to be graphics that include lots of little tiled figures used in video games. How does a browser know to split them up into separate images that pages can then call out individually? Because you tell him using position offsets in CSS. Top three hits for "css sprites": http://www.alistapart.com/articles/sprites http://css-tricks.com/css-sprites/ http://spritegen.website-performance.org/ Sprites are useful for collections of related of the same size and purpose. -- Michael Ludwig
Re: RFC: Apache2::CloseKeepAlive
Michael Ludwig wrote: Sprites are useful for collections of related of the same size and purpose. You don't have to restrict them to images of the same size. You can use a sprited image to replace any fixed size image, and any background image that repeats in one direction or the other (but not both). One caveat about that though is that background images don't print in most (maybe all) current browsers. In experiments with sprites and changes to some JS, we've managed to reduce render times on our websites by nearly half, and reduced the number of elements retrieved from the page to about a quarter of what they used to be in some cases. Adam
Re: RFC: Apache2::CloseKeepAlive
Thanks for your comments. They lead to lots to think about and try. I looked up "sprite sheets" and they seem to be graphics that include lots of little tiled figures used in video games. How does a browser know to split them up into separate images that pages can then call out individually? Ask Google ;) http://www.websiteoptimization.com/speed/tweak/css-sprites/ You should have said "CSS sprites" rather than "sprite sheets". The techniques cited look much harder to use than CloseKeepAlive, plus the comments at the end of the article say they don't work on MSIE, or (per another comment) on MSIE < 8. Looks like a technique that needs a new CPAN module to help mere mortals use it. Something to keep you out of mischief? Anyway I guess I'll muddle through with submitting CloseKeepAlive to CPAN. (This list has been quite helpful as to how to do that.) Obviously it's not for you, but maybe it can be useful to people less technologically advanced than you. Or as an example of how to use IPC::MMA. Thanks again, cmac
Re: RFC: Apache2::CloseKeepAlive
On Thu, Aug 27, 2009 at 5:20 PM, wrote: > Use a CDN and cache static content forever -- and rarely hit your serve. > In the end, each "page view" should only be one request to your mod_perl > server anyway. > > My site uses all defined headers to encourage proxies and browsers to cache > files. The performance of a site is the cross-product of all of the things > that it does to improve performance. > By the way, You might also want to look at something like S3 to server your static content. If you are low traffic (with potential spikes) then it will be much cheaper than a "real" CDN. Here's one tutorial: http://net.tutsplus.com/misc/use-amazon-s3-firefox-to-serve-static-files/ -- Bill Moseley mose...@hank.org
Re: RFC: Apache2::CloseKeepAlive
Adam Prime schrieb: Michael Ludwig wrote: Sprites are useful for collections of related of the same size and purpose. You don't have to restrict them to images of the same size. You can use a sprited image to replace any fixed size image, and any background image that repeats in one direction or the other (but not both). I thought that having the images tiled up regularly would make the CSS offsets easy to calculate. You'd only need the sequence number of the image and the sprite layout (say, 10 x 10) to calculate the offsets. But you appear to have had more exposure to sprites than me, so you're probably right. -- Michael Ludwig