Re: distributing software built on mod_perl
Whenever I creating shrink-wrapped software I always make packaging and distribution part of the development, qa and testing process. All packages for the platforms that we will be supporting. The reason I do this is to cut down on the customer support overhead. I've found you get less calls and emails regarding installation. And more importantly when it comes to supporting the customer they all have the same installation per platform which often times makes things much easier to debug (or at least removes alot of potential sources for problems). In practice what this means that we have an additional make target, package which creates the appropriate package for the platform it's running on. I also have automated unit test which pull apart the installation verifying meta data and install bits, as well as running the installation in a sandbox to verify installation occurs without issue. It can be alot of upfront time to set this up, but very little maintenance is required once it's up and running. YMMV -wjt
a better way to recognize module changes
I'm thinking about an improved solution to recognizing module changes in a running server, without restarting the server. These are the solutions I know about: 1) Apache2::Reload / Module::Reload These check whether modules have changed on each request, and if so, clear their symbols and reload them inside the process. Problem: some modules fail to reload properly. Sometimes the failure is intermittent, depending on the order of module loading and other esoteric details. Moose and ORM modules seem particularly prone to reload failures. For me, this level of unpredictability makes *::Reload too frustrating to use. 2) Catalyst auto-restart Catalyst has an engine (Catalyst::Engine::HTTP::Prefork::Restarter) which forks off a watcher process that waits for your modules to change. When they change, it restarts the server. The usual effect is that, between the time you hit save in your editor and reload your page, the server has restarted or at least begun restarting. Problems: Doesn't work well if you make a few changes in a row; the restart only captures your first change. Bad user experience if there's an error in your module; you have to realize the server has died, find the error message in some shell or log, and manually start up the server again. 3) Perrin's MaxRequestsPerChild=1 Perrin recently alerted me to the MaxRequestsPerChild=1 technique. That is, set MaxRequestsPerChild to 1, then load any potentially- changing modules in the *child*, not the parent (obviously only for development environments). Each request will hit a fresh child server, which will load all of your potentially-changing modules anew. This is the nicest solution I've seen so far. The only problem I can see is its performance - each potentially-changing module has to be loaded on each request. ** 4) My idea: Combine 2 and 3 As in 3, load any potentially-changing modules in the child. Leave MaxRequestsPerChild alone. As in 2, fork off a watcher process that waits for your modules to change. When they change, kill all the server's children explicitly. The end result is that you get reasonable performance when your modules don't change (e.g. when you are only futzing with templates), but when modules do change, you should see the effects immediately. This should be able to work with mod_perl, fastcgi, Net::Server, etc., as long as the parent server responds appropriately to the killing of all its children (by launching new ones). Apache, at least, seems to be ok with this. What do people think? Is this worth codifying in a module, or does something like this already exist? Thanks for any feedback Jon ** - You can try to load things only on demand, but often mod_perl code is written without 'use' statements as it assumes everything is loaded in the parent. You can also try to minimize the number of potentially-changing modules, but then you run the risk of leaving something off and having to adjust it and restart.
Re: a better way to recognize module changes
On Fri, Sep 11, 2009 at 5:26 PM, Jonathan Swartz swa...@pobox.com wrote: This is the nicest solution I've seen so far. The only problem I can see is its performance - each potentially-changing module has to be loaded on each request. ** How long does it take for you? I've run a lot of large mod_perl apps this way and never seen it take more than 3 seconds to compile and generate a page. Maybe this is only an issue with Moose? Or maybe there's some expensive initialization that could be skipped in development? Anyway, another approach would be to set MaxClients to 1 and install cleanup handler that kills the current process if it sees that any of the watched modules have been changed. Then you don't need a separate process. - Perrin
Re: a better way to recognize module changes
Maybe somebody can refute what I'm seeing, but as of mod_perl-2.0.4, Apache2::Reload is gone (so you can remove that from your list of options). -- Devin On Fri, 2009-09-11 at 14:26 -0700, Jonathan Swartz wrote: I'm thinking about an improved solution to recognizing module changes in a running server, without restarting the server. These are the solutions I know about: 1) Apache2::Reload / Module::Reload These check whether modules have changed on each request, and if so, clear their symbols and reload them inside the process. Problem: some modules fail to reload properly. Sometimes the failure is intermittent, depending on the order of module loading and other esoteric details. Moose and ORM modules seem particularly prone to reload failures. For me, this level of unpredictability makes *::Reload too frustrating to use. 2) Catalyst auto-restart Catalyst has an engine (Catalyst::Engine::HTTP::Prefork::Restarter) which forks off a watcher process that waits for your modules to change. When they change, it restarts the server. The usual effect is that, between the time you hit save in your editor and reload your page, the server has restarted or at least begun restarting. Problems: Doesn't work well if you make a few changes in a row; the restart only captures your first change. Bad user experience if there's an error in your module; you have to realize the server has died, find the error message in some shell or log, and manually start up the server again. 3) Perrin's MaxRequestsPerChild=1 Perrin recently alerted me to the MaxRequestsPerChild=1 technique. That is, set MaxRequestsPerChild to 1, then load any potentially- changing modules in the *child*, not the parent (obviously only for development environments). Each request will hit a fresh child server, which will load all of your potentially-changing modules anew. This is the nicest solution I've seen so far. The only problem I can see is its performance - each potentially-changing module has to be loaded on each request. ** 4) My idea: Combine 2 and 3 As in 3, load any potentially-changing modules in the child. Leave MaxRequestsPerChild alone. As in 2, fork off a watcher process that waits for your modules to change. When they change, kill all the server's children explicitly. The end result is that you get reasonable performance when your modules don't change (e.g. when you are only futzing with templates), but when modules do change, you should see the effects immediately. This should be able to work with mod_perl, fastcgi, Net::Server, etc., as long as the parent server responds appropriately to the killing of all its children (by launching new ones). Apache, at least, seems to be ok with this. What do people think? Is this worth codifying in a module, or does something like this already exist? Thanks for any feedback Jon ** - You can try to load things only on demand, but often mod_perl code is written without 'use' statements as it assumes everything is loaded in the parent. You can also try to minimize the number of potentially-changing modules, but then you run the risk of leaving something off and having to adjust it and restart. -- Cheers, Devin Teske - CONTACT INFORMATION - Field Engineer Metavante Corporation 626-573-6040 Office 510-735-5650 Mobile devin.te...@metavante.com - LEGAL DISCLAIMER - This message contains confidential and proprietary information of the sender, and is intended only for the person(s) to whom it is addressed. Any use, distribution, copying or disclosure by any other person is strictly prohibited. If you have received this message in error, please notify the e-mail sender immediately, and delete the original message without making a copy. - END TRANSMISSION -
Re: a better way to recognize module changes
On Fri, Sep 11, 2009 at 3:02 PM, Devin Teske dte...@vicor.com wrote: Maybe somebody can refute what I'm seeing, but as of mod_perl-2.0.4, Apache2::Reload is gone (so you can remove that from your list of options). It was not bundled with 2.0.4 but is available on CPAN: http://search.cpan.org/dist/Apache-Reload It will be bundled again with 2.0.5. There was a discussion on the dev list about how to manage perl based Apache2?::* modules with mod_perl core releases and the specific version control procedures to accomplish this in a pain free manner. -- Devin On Fri, 2009-09-11 at 14:26 -0700, Jonathan Swartz wrote: I'm thinking about an improved solution to recognizing module changes in a running server, without restarting the server. These are the solutions I know about: 1) Apache2::Reload / Module::Reload These check whether modules have changed on each request, and if so, clear their symbols and reload them inside the process. Problem: some modules fail to reload properly. Sometimes the failure is intermittent, depending on the order of module loading and other esoteric details. Moose and ORM modules seem particularly prone to reload failures. For me, this level of unpredictability makes *::Reload too frustrating to use. 2) Catalyst auto-restart Catalyst has an engine (Catalyst::Engine::HTTP::Prefork::Restarter) which forks off a watcher process that waits for your modules to change. When they change, it restarts the server. The usual effect is that, between the time you hit save in your editor and reload your page, the server has restarted or at least begun restarting. Problems: Doesn't work well if you make a few changes in a row; the restart only captures your first change. Bad user experience if there's an error in your module; you have to realize the server has died, find the error message in some shell or log, and manually start up the server again. 3) Perrin's MaxRequestsPerChild=1 Perrin recently alerted me to the MaxRequestsPerChild=1 technique. That is, set MaxRequestsPerChild to 1, then load any potentially- changing modules in the *child*, not the parent (obviously only for development environments). Each request will hit a fresh child server, which will load all of your potentially-changing modules anew. This is the nicest solution I've seen so far. The only problem I can see is its performance - each potentially-changing module has to be loaded on each request. ** 4) My idea: Combine 2 and 3 As in 3, load any potentially-changing modules in the child. Leave MaxRequestsPerChild alone. As in 2, fork off a watcher process that waits for your modules to change. When they change, kill all the server's children explicitly. The end result is that you get reasonable performance when your modules don't change (e.g. when you are only futzing with templates), but when modules do change, you should see the effects immediately. This should be able to work with mod_perl, fastcgi, Net::Server, etc., as long as the parent server responds appropriately to the killing of all its children (by launching new ones). Apache, at least, seems to be ok with this. What do people think? Is this worth codifying in a module, or does something like this already exist? Thanks for any feedback Jon ** - You can try to load things only on demand, but often mod_perl code is written without 'use' statements as it assumes everything is loaded in the parent. You can also try to minimize the number of potentially-changing modules, but then you run the risk of leaving something off and having to adjust it and restart. -- Cheers, Devin Teske - CONTACT INFORMATION - Field Engineer Metavante Corporation 626-573-6040 Office 510-735-5650 Mobile devin.te...@metavante.com - LEGAL DISCLAIMER - This message contains confidential and proprietary information of the sender, and is intended only for the person(s) to whom it is addressed. Any use, distribution, copying or disclosure by any other person is strictly prohibited. If you have received this message in error, please notify the e-mail sender immediately, and delete the original message without making a copy. - END TRANSMISSION -
Re: a better way to recognize module changes
It seems like it's available separately in Apache-Reload distribution: http://cpan.uwinnipeg.ca/dist/Apache-Reload But it's already pretty much a straw-man option for me. :) Jon On Sep 11, 2009, at 3:02 PM, Devin Teske wrote: Maybe somebody can refute what I'm seeing, but as of mod_perl-2.0.4, Apache2::Reload is gone (so you can remove that from your list of options). -- Devin On Fri, 2009-09-11 at 14:26 -0700, Jonathan Swartz wrote: I'm thinking about an improved solution to recognizing module changes in a running server, without restarting the server. These are the solutions I know about: 1) Apache2::Reload / Module::Reload These check whether modules have changed on each request, and if so, clear their symbols and reload them inside the process. Problem: some modules fail to reload properly. Sometimes the failure is intermittent, depending on the order of module loading and other esoteric details. Moose and ORM modules seem particularly prone to reload failures. For me, this level of unpredictability makes *::Reload too frustrating to use. 2) Catalyst auto-restart Catalyst has an engine (Catalyst::Engine::HTTP::Prefork::Restarter) which forks off a watcher process that waits for your modules to change. When they change, it restarts the server. The usual effect is that, between the time you hit save in your editor and reload your page, the server has restarted or at least begun restarting. Problems: Doesn't work well if you make a few changes in a row; the restart only captures your first change. Bad user experience if there's an error in your module; you have to realize the server has died, find the error message in some shell or log, and manually start up the server again. 3) Perrin's MaxRequestsPerChild=1 Perrin recently alerted me to the MaxRequestsPerChild=1 technique. That is, set MaxRequestsPerChild to 1, then load any potentially- changing modules in the *child*, not the parent (obviously only for development environments). Each request will hit a fresh child server, which will load all of your potentially-changing modules anew. This is the nicest solution I've seen so far. The only problem I can see is its performance - each potentially-changing module has to be loaded on each request. ** 4) My idea: Combine 2 and 3 As in 3, load any potentially-changing modules in the child. Leave MaxRequestsPerChild alone. As in 2, fork off a watcher process that waits for your modules to change. When they change, kill all the server's children explicitly. The end result is that you get reasonable performance when your modules don't change (e.g. when you are only futzing with templates), but when modules do change, you should see the effects immediately. This should be able to work with mod_perl, fastcgi, Net::Server, etc., as long as the parent server responds appropriately to the killing of all its children (by launching new ones). Apache, at least, seems to be ok with this. What do people think? Is this worth codifying in a module, or does something like this already exist? Thanks for any feedback Jon ** - You can try to load things only on demand, but often mod_perl code is written without 'use' statements as it assumes everything is loaded in the parent. You can also try to minimize the number of potentially-changing modules, but then you run the risk of leaving something off and having to adjust it and restart. -- Cheers, Devin Teske - CONTACT INFORMATION - Field Engineer Metavante Corporation 626-573-6040 Office 510-735-5650 Mobile devin.te...@metavante.com - LEGAL DISCLAIMER - This message contains confidential and proprietary information of the sender, and is intended only for the person(s) to whom it is addressed. Any use, distribution, copying or disclosure by any other person is strictly prohibited. If you have received this message in error, please notify the e-mail sender immediately, and delete the original message without making a copy. - END TRANSMISSION -
Re: a better way to recognize module changes
On Sep 11, 2009, at 2:52 PM, Perrin Harkins wrote: On Fri, Sep 11, 2009 at 5:26 PM, Jonathan Swartz swa...@pobox.com wrote: This is the nicest solution I've seen so far. The only problem I can see is its performance - each potentially-changing module has to be loaded on each request. ** How long does it take for you? I've run a lot of large mod_perl apps this way and never seen it take more than 3 seconds to compile and generate a page. Maybe this is only an issue with Moose? Or maybe there's some expensive initialization that could be skipped in development? It can take a few seconds to load all our modules (sans the CPAN modules they use). Perhaps I'm particularly sensitive to development environment efficiency. But if we could codify a relatively simple solution that eliminates those seconds, why not? Anyway, another approach would be to set MaxClients to 1 and install cleanup handler that kills the current process if it sees that any of the watched modules have been changed. Then you don't need a separate process. But if you change a module, wouldn't your first subsequent request hit the 'old' code? That's the benefit of an independent watcher, it'll operate in the time between when you've changed your module and when you hit the server again. Jon
Re: a better way to recognize module changes
Was there any particular reason that it wasn't packaged with 2.0.4? (meaning, can I just supplant the one from 2.0.3... or was it removed only to be re-added after being patched for some particular purpose?) -- Devin On Fri, 2009-09-11 at 15:11 -0700, Fred Moyer wrote: On Fri, Sep 11, 2009 at 3:02 PM, Devin Teske dte...@vicor.com wrote: Maybe somebody can refute what I'm seeing, but as of mod_perl-2.0.4, Apache2::Reload is gone (so you can remove that from your list of options). It was not bundled with 2.0.4 but is available on CPAN: http://search.cpan.org/dist/Apache-Reload It will be bundled again with 2.0.5. There was a discussion on the dev list about how to manage perl based Apache2?::* modules with mod_perl core releases and the specific version control procedures to accomplish this in a pain free manner. -- Devin On Fri, 2009-09-11 at 14:26 -0700, Jonathan Swartz wrote: I'm thinking about an improved solution to recognizing module changes in a running server, without restarting the server. These are the solutions I know about: 1) Apache2::Reload / Module::Reload These check whether modules have changed on each request, and if so, clear their symbols and reload them inside the process. Problem: some modules fail to reload properly. Sometimes the failure is intermittent, depending on the order of module loading and other esoteric details. Moose and ORM modules seem particularly prone to reload failures. For me, this level of unpredictability makes *::Reload too frustrating to use. 2) Catalyst auto-restart Catalyst has an engine (Catalyst::Engine::HTTP::Prefork::Restarter) which forks off a watcher process that waits for your modules to change. When they change, it restarts the server. The usual effect is that, between the time you hit save in your editor and reload your page, the server has restarted or at least begun restarting. Problems: Doesn't work well if you make a few changes in a row; the restart only captures your first change. Bad user experience if there's an error in your module; you have to realize the server has died, find the error message in some shell or log, and manually start up the server again. 3) Perrin's MaxRequestsPerChild=1 Perrin recently alerted me to the MaxRequestsPerChild=1 technique. That is, set MaxRequestsPerChild to 1, then load any potentially- changing modules in the *child*, not the parent (obviously only for development environments). Each request will hit a fresh child server, which will load all of your potentially-changing modules anew. This is the nicest solution I've seen so far. The only problem I can see is its performance - each potentially-changing module has to be loaded on each request. ** 4) My idea: Combine 2 and 3 As in 3, load any potentially-changing modules in the child. Leave MaxRequestsPerChild alone. As in 2, fork off a watcher process that waits for your modules to change. When they change, kill all the server's children explicitly. The end result is that you get reasonable performance when your modules don't change (e.g. when you are only futzing with templates), but when modules do change, you should see the effects immediately. This should be able to work with mod_perl, fastcgi, Net::Server, etc., as long as the parent server responds appropriately to the killing of all its children (by launching new ones). Apache, at least, seems to be ok with this. What do people think? Is this worth codifying in a module, or does something like this already exist? Thanks for any feedback Jon ** - You can try to load things only on demand, but often mod_perl code is written without 'use' statements as it assumes everything is loaded in the parent. You can also try to minimize the number of potentially-changing modules, but then you run the risk of leaving something off and having to adjust it and restart. -- Cheers, Devin Teske - CONTACT INFORMATION - Field Engineer Metavante Corporation 626-573-6040 Office 510-735-5650 Mobile devin.te...@metavante.com - LEGAL DISCLAIMER - This message contains confidential and proprietary information of the sender, and is intended only for the person(s) to whom it is addressed. Any use, distribution, copying or disclosure by any other person is strictly prohibited. If you have received this message in error, please notify the e-mail sender immediately, and delete the original message without making a copy. - END TRANSMISSION - -- Cheers, Devin Teske - CONTACT INFORMATION - Field Engineer Metavante Corporation 626-573-6040 Office 510-735-5650 Mobile devin.te...@metavante.com - LEGAL DISCLAIMER - This message contains confidential and proprietary information of the sender, and is intended only for the person(s) to whom it is addressed. Any use, distribution, copying or disclosure by any
Re: a better way to recognize module changes
Incidentally Perrin - how do you come up with the list of vendor (i.e. not your project's) modules to load in the parent? Do you just load a page, look at %INC, and then subtract out your personal modules? Do you have to do this every so often to catch new vendor modules that have snuck in as dependencies? On Sep 11, 2009, at 2:52 PM, Perrin Harkins wrote: On Fri, Sep 11, 2009 at 5:26 PM, Jonathan Swartz swa...@pobox.com wrote: This is the nicest solution I've seen so far. The only problem I can see is its performance - each potentially-changing module has to be loaded on each request. ** How long does it take for you? I've run a lot of large mod_perl apps this way and never seen it take more than 3 seconds to compile and generate a page. Maybe this is only an issue with Moose? Or maybe there's some expensive initialization that could be skipped in development? Anyway, another approach would be to set MaxClients to 1 and install cleanup handler that kills the current process if it sees that any of the watched modules have been changed. Then you don't need a separate process. - Perrin
Re: a better way to recognize module changes
On Fri, Sep 11, 2009 at 3:20 PM, Devin Teske dte...@vicor.com wrote: Was there any particular reason that it wasn't packaged with 2.0.4? (meaning, can I just supplant the one from 2.0.3... or was it removed only to be re-added after being patched for some particular purpose?) It was not included because of a miscommunication when releasing 0.10. You can install 0.10 from CPAN and should be fine. The conclusions after the post mortem were that the latest stable versions of Apache::* modules should be included with mod_perl core releases, but later versions of those modules should be easily upgradeable from CPAN. -- Devin On Fri, 2009-09-11 at 15:11 -0700, Fred Moyer wrote: On Fri, Sep 11, 2009 at 3:02 PM, Devin Teske dte...@vicor.com wrote: Maybe somebody can refute what I'm seeing, but as of mod_perl-2.0.4, Apache2::Reload is gone (so you can remove that from your list of options). It was not bundled with 2.0.4 but is available on CPAN: http://search.cpan.org/dist/Apache-Reload It will be bundled again with 2.0.5. There was a discussion on the dev list about how to manage perl based Apache2?::* modules with mod_perl core releases and the specific version control procedures to accomplish this in a pain free manner. -- Devin On Fri, 2009-09-11 at 14:26 -0700, Jonathan Swartz wrote: I'm thinking about an improved solution to recognizing module changes in a running server, without restarting the server. These are the solutions I know about: 1) Apache2::Reload / Module::Reload These check whether modules have changed on each request, and if so, clear their symbols and reload them inside the process. Problem: some modules fail to reload properly. Sometimes the failure is intermittent, depending on the order of module loading and other esoteric details. Moose and ORM modules seem particularly prone to reload failures. For me, this level of unpredictability makes *::Reload too frustrating to use. 2) Catalyst auto-restart Catalyst has an engine (Catalyst::Engine::HTTP::Prefork::Restarter) which forks off a watcher process that waits for your modules to change. When they change, it restarts the server. The usual effect is that, between the time you hit save in your editor and reload your page, the server has restarted or at least begun restarting. Problems: Doesn't work well if you make a few changes in a row; the restart only captures your first change. Bad user experience if there's an error in your module; you have to realize the server has died, find the error message in some shell or log, and manually start up the server again. 3) Perrin's MaxRequestsPerChild=1 Perrin recently alerted me to the MaxRequestsPerChild=1 technique. That is, set MaxRequestsPerChild to 1, then load any potentially- changing modules in the *child*, not the parent (obviously only for development environments). Each request will hit a fresh child server, which will load all of your potentially-changing modules anew. This is the nicest solution I've seen so far. The only problem I can see is its performance - each potentially-changing module has to be loaded on each request. ** 4) My idea: Combine 2 and 3 As in 3, load any potentially-changing modules in the child. Leave MaxRequestsPerChild alone. As in 2, fork off a watcher process that waits for your modules to change. When they change, kill all the server's children explicitly. The end result is that you get reasonable performance when your modules don't change (e.g. when you are only futzing with templates), but when modules do change, you should see the effects immediately. This should be able to work with mod_perl, fastcgi, Net::Server, etc., as long as the parent server responds appropriately to the killing of all its children (by launching new ones). Apache, at least, seems to be ok with this. What do people think? Is this worth codifying in a module, or does something like this already exist? Thanks for any feedback Jon ** - You can try to load things only on demand, but often mod_perl code is written without 'use' statements as it assumes everything is loaded in the parent. You can also try to minimize the number of potentially-changing modules, but then you run the risk of leaving something off and having to adjust it and restart. -- Cheers, Devin Teske - CONTACT INFORMATION - Field Engineer Metavante Corporation 626-573-6040 Office 510-735-5650 Mobile devin.te...@metavante.com - LEGAL DISCLAIMER - This message contains confidential and proprietary information of the sender, and is intended only for the person(s) to whom it is addressed. Any use, distribution, copying or disclosure by any other person is strictly prohibited. If you have received this message in error, please notify the e-mail sender immediately, and delete the original message without
Re: a better way to recognize module changes
On Fri, Sep 11, 2009 at 3:11 PM, Jonathan Swartz swa...@pobox.com wrote: It seems like it's available separately in Apache-Reload distribution: http://cpan.uwinnipeg.ca/dist/Apache-Reload But it's already pretty much a straw-man option for me. :) Problem: some modules fail to reload properly. Sometimes the failure is intermittent, depending on the order of module loading and other esoteric details. Moose and ORM modules seem particularly prone to reload failures. For me, this level of unpredictability makes *::Reload too frustrating to use. I've run into issues with reloading ORM based modules such as when I change my DBIx::Class based schema. I get an inconsistent hierarchy error from Class::C3. So in those cases a server restart is required. However, if your model layer is abstracted from mod_perl, you should be able to develop against your model layer using tests, and then install your upgraded model and restart Apache then. The behavior where I encounter problems reloading changes to ORM modules has enforced a good separation of model (ORM) and controller( mod_perl ) code in my application. Jon On Sep 11, 2009, at 3:02 PM, Devin Teske wrote: Maybe somebody can refute what I'm seeing, but as of mod_perl-2.0.4, Apache2::Reload is gone (so you can remove that from your list of options). -- Devin On Fri, 2009-09-11 at 14:26 -0700, Jonathan Swartz wrote: I'm thinking about an improved solution to recognizing module changes in a running server, without restarting the server. These are the solutions I know about: 1) Apache2::Reload / Module::Reload These check whether modules have changed on each request, and if so, clear their symbols and reload them inside the process. Problem: some modules fail to reload properly. Sometimes the failure is intermittent, depending on the order of module loading and other esoteric details. Moose and ORM modules seem particularly prone to reload failures. For me, this level of unpredictability makes *::Reload too frustrating to use. 2) Catalyst auto-restart Catalyst has an engine (Catalyst::Engine::HTTP::Prefork::Restarter) which forks off a watcher process that waits for your modules to change. When they change, it restarts the server. The usual effect is that, between the time you hit save in your editor and reload your page, the server has restarted or at least begun restarting. Problems: Doesn't work well if you make a few changes in a row; the restart only captures your first change. Bad user experience if there's an error in your module; you have to realize the server has died, find the error message in some shell or log, and manually start up the server again. 3) Perrin's MaxRequestsPerChild=1 Perrin recently alerted me to the MaxRequestsPerChild=1 technique. That is, set MaxRequestsPerChild to 1, then load any potentially- changing modules in the *child*, not the parent (obviously only for development environments). Each request will hit a fresh child server, which will load all of your potentially-changing modules anew. This is the nicest solution I've seen so far. The only problem I can see is its performance - each potentially-changing module has to be loaded on each request. ** 4) My idea: Combine 2 and 3 As in 3, load any potentially-changing modules in the child. Leave MaxRequestsPerChild alone. As in 2, fork off a watcher process that waits for your modules to change. When they change, kill all the server's children explicitly. The end result is that you get reasonable performance when your modules don't change (e.g. when you are only futzing with templates), but when modules do change, you should see the effects immediately. This should be able to work with mod_perl, fastcgi, Net::Server, etc., as long as the parent server responds appropriately to the killing of all its children (by launching new ones). Apache, at least, seems to be ok with this. What do people think? Is this worth codifying in a module, or does something like this already exist? Thanks for any feedback Jon ** - You can try to load things only on demand, but often mod_perl code is written without 'use' statements as it assumes everything is loaded in the parent. You can also try to minimize the number of potentially-changing modules, but then you run the risk of leaving something off and having to adjust it and restart. -- Cheers, Devin Teske - CONTACT INFORMATION - Field Engineer Metavante Corporation 626-573-6040 Office 510-735-5650 Mobile devin.te...@metavante.com - LEGAL DISCLAIMER - This message contains confidential and proprietary information of the sender, and is intended only for the person(s) to whom it is addressed. Any use, distribution, copying or disclosure by any other person is strictly prohibited. If you have received this message in error, please notify the e-mail sender immediately, and delete the original message without making a copy. - END
Re: a better way to recognize module changes
On Fri, Sep 11, 2009 at 3:11 PM, Jonathan Swartz swa...@pobox.com wrote: It seems like it's available separately in Apache-Reload distribution: http://cpan.uwinnipeg.ca/dist/Apache-Reload But it's already pretty much a straw-man option for me. :) Problem: some modules fail to reload properly. Sometimes the failure is intermittent, depending on the order of module loading and other esoteric details. Moose and ORM modules seem particularly prone to reload failures. For me, this level of unpredictability makes *::Reload too frustrating to use. I've run into issues with reloading ORM based modules such as when I change my DBIx::Class based schema. I get an inconsistent hierarchy error from Class::C3. So in those cases a server restart is required. However, if your model layer is abstracted from mod_perl, you should be able to develop against your model layer using tests, and then install your upgraded model and restart Apache then. The behavior where I encounter problems reloading changes to ORM modules has enforced a good separation of model (ORM) and controller( mod_perl ) code in my application. I'm all for TDD, to be sure, but in practice not everything - especially with controller and view-specific modules - is going to be developed in this way. I've often exhorted Mason developers to move things out of components into modules, but a real objection to that has been the server restart cost, and I've never had a good answer for it. Mason components (like TT templates, etc.) can pretty much always be reloaded without restart, which I expect is one reason why so much code ends up there. Jon
Re: a better way to recognize module changes
On Fri, Sep 11, 2009 at 3:37 PM, Jonathan Swartz swa...@pobox.com wrote: On Fri, Sep 11, 2009 at 3:11 PM, Jonathan Swartz swa...@pobox.com wrote: It seems like it's available separately in Apache-Reload distribution: http://cpan.uwinnipeg.ca/dist/Apache-Reload But it's already pretty much a straw-man option for me. :) Problem: some modules fail to reload properly. Sometimes the failure is intermittent, depending on the order of module loading and other esoteric details. Moose and ORM modules seem particularly prone to reload failures. For me, this level of unpredictability makes *::Reload too frustrating to use. I've run into issues with reloading ORM based modules such as when I change my DBIx::Class based schema. I get an inconsistent hierarchy error from Class::C3. So in those cases a server restart is required. However, if your model layer is abstracted from mod_perl, you should be able to develop against your model layer using tests, and then install your upgraded model and restart Apache then. The behavior where I encounter problems reloading changes to ORM modules has enforced a good separation of model (ORM) and controller( mod_perl ) code in my application. I'm all for TDD, to be sure, but in practice not everything - especially with controller and view-specific modules - is going to be developed in this way. I've often exhorted Mason developers to move things out of components into modules, but a real objection to that has been the server restart cost, and I've never had a good answer for it. Mason components (like TT templates, etc.) can pretty much always be reloaded without restart, which I expect is one reason why so much code ends up there. That makes a lot of sense. Only thing I could suggest would be to exclude your Moose based modules from Apache::Reload reload list if they are causing problems. Although, if there was a way to kill off user owned httpd processes when a code change was made, you could extend Apache::Reload to have certain modules cause httpd recycling vs code reloading. Maybe something like: PerlSetVar ReloadAll On PerlSetVar ReloadHttpds My::Moose So that modules in ReloadHttpds terminates existing user httpd processes and causes the server to fork off new httpd children. Jon
Re: a better way to recognize module changes
On Sep 11, 2009, at 3:53 PM, Fred Moyer wrote: On Fri, Sep 11, 2009 at 3:37 PM, Jonathan Swartz swa...@pobox.com wrote: On Fri, Sep 11, 2009 at 3:11 PM, Jonathan Swartz swa...@pobox.com wrote: It seems like it's available separately in Apache-Reload distribution: http://cpan.uwinnipeg.ca/dist/Apache-Reload But it's already pretty much a straw-man option for me. :) Problem: some modules fail to reload properly. Sometimes the failure is intermittent, depending on the order of module loading and other esoteric details. Moose and ORM modules seem particularly prone to reload failures. For me, this level of unpredictability makes *::Reload too frustrating to use. I've run into issues with reloading ORM based modules such as when I change my DBIx::Class based schema. I get an inconsistent hierarchy error from Class::C3. So in those cases a server restart is required. However, if your model layer is abstracted from mod_perl, you should be able to develop against your model layer using tests, and then install your upgraded model and restart Apache then. The behavior where I encounter problems reloading changes to ORM modules has enforced a good separation of model (ORM) and controller( mod_perl ) code in my application. I'm all for TDD, to be sure, but in practice not everything - especially with controller and view-specific modules - is going to be developed in this way. I've often exhorted Mason developers to move things out of components into modules, but a real objection to that has been the server restart cost, and I've never had a good answer for it. Mason components (like TT templates, etc.) can pretty much always be reloaded without restart, which I expect is one reason why so much code ends up there. That makes a lot of sense. Only thing I could suggest would be to exclude your Moose based modules from Apache::Reload reload list if they are causing problems. Although, if there was a way to kill off user owned httpd processes when a code change was made, you could extend Apache::Reload to have certain modules cause httpd recycling vs code reloading. Maybe something like: PerlSetVar ReloadAll On PerlSetVar ReloadHttpds My::Moose So that modules in ReloadHttpds terminates existing user httpd processes and causes the server to fork off new httpd children. But again, if Apache::Reload runs as part of the child process itself, how can it implement ReloadHttpds effectively? It would have to kill itself off, which would mean your first request would generate an error - not a good user experience.
Re: a better way to recognize module changes
On 9/11/09 6:57 PM, Jonathan Swartz wrote: PerlSetVar ReloadAll On PerlSetVar ReloadHttpds My::Moose So that modules in ReloadHttpds terminates existing user httpd processes and causes the server to fork off new httpd children. But again, if Apache::Reload runs as part of the child process itself, how can it implement ReloadHttpds effectively? It would have to kill itself off, which would mean your first request would generate an error - not a good user experience. The last thing I did in this area looked something like this: PerlCleanupHandler My::WebApp::Restarter PerlSetVar WatcherInterval 60 PerlSetVar WatcherCompileCheck 1 where all the handler did was fork off a watcher process if one didn't already exist. The watcher would then stat() all the files in %INC periodically. If one had changed, it'd attempt to compile it (if WatcherCompileCheck was true) catching/absorbing any errors to prevent ugly output, and then restart the apache server by sending it a signal if the file compiled okay. Doing a full restart avoids all the odd module re-load issues (and there are many). The polling interval may seem too long, but this would all happen as you were working on the code and before you reached back to a browser window to reload a page. By the time you did, the hope was that the server would already be back up again. One obvious improvement would be to use kqueue/inotify/FSEvents to avoid polling entirely, making it much more likely that the server would be back in time for your next request. -John
FW: Apache::DBI Failed due to +GlobalRequest
Hi, I have configured with my apache 2.2 with Mod perl. When I tried to use Apache::DBI for persistent Database connection, It shows the below error [Thu Sep 10 14:39:30 2009] [error] Global $r object is not available. Set:\n\tPerlOptions +GlobalRequest\nin httpd.conf at /var/www/audashboard/exec/startup.pl line 24.\nCompilation failed in require at (eval 5) line 1.\n [Thu Sep 10 14:40:05 2009] [warn] Init: Session Cache is not configured [hint: SSLSessionCache] [Thu Sep 10 14:40:05 2009] [error] Global symbol $r requires explicit package name at startup.pl line 24.\nCompilation failed in require at (eval 5) line 1.\n I have re-installed CGI 3.45 and enable the PerlOptions in my virtual host. But still it is not working for me . Can you please suggest me on this ? Raja
