Stephan Richter wrote:
On Tuesday 09 May 2006 07:22, Jim Fulton wrote:
I guess we need to make this a priority for the next release.
Python simply does not support a general robust reload, other than
restart.
I think that there are 2 ways we can make progress in this area:
- Speed up restart. I think there are a lot of ways that restarts
can be made faster:
o Optimizae what we're doing now. I suspect that there are some
opportunuties here.
I have applied for the SoC with a proposal to enhance ZCML. My proposal is
attached. It discusses some of the optimization options we talked of before.
...
I propose to allow local components to be configurable through ZCML. This
goal became feasible with the recent component architecture refactorings by
Jim Fulton. Any site (local or global) can now have a set of base sites that
are used to provide additional components. I want to allow ZCML to specify
any number of base sites and add components to it. Here is an example of how
I imagine it to look like (ZCML)::
<site name="my-base-site" />
I'll note that I'd like site-name to be a convenience directive that is
equivalent to defining a GlobalComponents object and registering it as
an IComponents utility. Note that GlobalComponents doesn't exist yet. :)
It should be very similar to BaseGlobalComponents, except that it pickles
and unpickles as a named utilities registered with a BaseGlobalComponents.
<configure use-site="my-base-site">
...
</configure>
The new ``site`` directive creates a new site. The ``configure`` directive
will grow an ``use-site`` attribute that specifies the site to put in the
components. By default,
Actually, the site to put the components in. :)
There will be a new registry of all ZCML-defined sites.
No! Use the base utility registry.
> All existing ZCML
directives have to be reviewed and it must be ensured that they are
multi-site aware. The tricky part of the implementation will be to hook in
those sites as bases to local sites. It must be ensured that the ZODB can
load having filesystem-based sub-sites, error handling must be carefully
considered and an UI must be written.
The pickling aspects are pretty trivial. I'm not sure what UI you are refering
to.
...
Another big problem in Zope 3 is the startup time. Some code profiling has
determined that most of the time in the startup process is lost in parsing,
converting and validating ZCML directives with their schemas. Thus, this
startup problem is not purely a Zope 3 problem, but one that affects
everyone using ZCML.
This problem can be addressed in several ways. The most obvious one to Zope
2 developers would be not to restart the application server, but only reload
the packages and modules that were affected by the code changes. This
approach has been used in Zope 2 for many years, but it several serious
problems and some of the smartest people I know have not been able to
completely solve the problems. Based on that, I do not think that a proposal
suggesting this approach would be accepted.
The second approach is to reduce the ZCML processing time, which could be
integrated into the reload mechanism for Zope 2. This can be accomplished by
storing some binary representation of the ZCML, similarly to ``*.pyc`` files
in Python. Again there are several choices to consider and they should
probably all be tried. The first solution would be to store a pickle of each
parsed directive, namely the action and its arguments. There would be one
pickle file fore each ZCML file. When the ZCML file changed, the pickle
would be updated. Pickle loading would be much faster than pure ZCML
loading, since no XML-parsing, value conversion and schema validation would
be necessary.
Note that this will require a refactoring of ZCML handlers to define picklable
actions. This will also require refactoring so that work now done by handlers
be defered to action execution.
On the other hand, ZCML creates actions that are eventually
executed. Actions are created by executing the directive handlers. Thus the
optimization in this approach would be even greater. The problem with this
approach is that not all directives are easily pickable. Directive handlers
often create new types/classes on the fly. This problem could be solved by
ensuring that directives only create pickable actions. Clearly, this would
require a lot more work, since you would have to go through all directives
to ensure their pickability and also provide fallbacks for 3rd-party
directives.
Yup. One possibility is to have a mechanism in which the pickle files
are created only when possible. That is, we try to pickle the actions
and give up if pickling fails. If we update the core directives to
work this way, that will account for most configuration files and
could provide a significant speedup even if not all configuration files
are handled. Note that, if this is pursued. then a prototype should
test the speedup by updating some core directives and measuring the
impact.
Overall, I believe that the performance of ZCML could be improved
drastically either way and that the benefit would be immediately visible for
anyone using ZCML, including Zope 2 developers using Zope 3 components via
Five.
IMO, the action pickling work should be deferred until we allow configuration
to be expressed in Python and assess the impacts. If many people switch to
defining configuration actions in Python, then speeding action generation from
XML could be moot.
Jim
--
Jim Fulton mailto:[EMAIL PROTECTED] Python Powered!
CTO (540) 361-1714 http://www.python.org
Zope Corporation http://www.zope.com http://www.zope.org
_______________________________________________
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
