Re: [Zope-dev] Re: straighting out the SQLAlchemy integration mess
On Fri, Apr 11, 2008 at 01:21:45PM +0200, Andreas Jung wrote: >> Unfortunately, it doesn't seem an option, right now, to avoid using ZCML >> to setup database connections. So I'm not sure how you want to implement >> your ideas? >> > > I have no need for ideas to be implemented. We're using SA within in > several large projects based on z3c.sqlalchemy. And nothing beats the > simplicity > of of an .ini file or an environment variable. I see. I suppose you could also write a zcml declaration to pluck out the database config from an environment variable and register a utility. >> In general, I think if you want your configuration expressed in ZConfig >> and available as a utility, you have a problem. >> >> I'm not sure of: >> * How do I register a utility in ZConfig? > > z3c.sqlalchemy provides a method for registering a SA connection as a named > utility. This code is usually called during the startup phase of Zope > within the related initialize() methds of our products. The DSN are > acquired from well-known environment variables...this just works - easy, > simple, reliable. > >> * How will the above registration work properly in functional tests >> (especially with tearing down and setting up layers)? > > Functional tests in our company are tests done by humans. Why? Our > applications are very complex with lots of dependencies. Not speaking of > lots of involved databases, big storages.. I try do what I can... which is relatively minimal. At least some components are thoroughly tested with the parts of the db schema they depend on. -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: straighting out the SQLAlchemy integration mess
On Friday 11 April 2008, Bernd Dorn wrote: > although it is a few lines of code to get db uris from somewhere else, > and this could also be application code if you use paster, you can stick options into your pater ini script. We are currently doing this for a setting in our app. Regards, Stephan -- Stephan Richter Web Software Design, Development and Training Google me. "Zope Stephan Richter" ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: straighting out the SQLAlchemy integration mess
Bernd Dorn wrote: [whether configuration should be in ZCML] although it is a few lines of code to get db uris from somewhere else, and this could also be application code. I'm not against ZCML, as long as those lines of code to get the db URI from application code also exist. A good API to configure things without ZCML is necessary (that the ZCML can then turn around and use - this is a good way to create ZCML directives anyway). A nice thing about collective.lead is that it doesn't introduce custom ZCML directives, so in Grok, using it means there's no ZCML at all and no need to write any grokkers for it either. If it did introduce custom directives to do the same thing, that's fine, as long the original way isn't damaged by it. Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: [zope.testing.doctest] a nasty surprise
Balazs Ree wrote at 2008-4-11 09:19 +0200:
> ...
>>> In order to analyse a difficult problem, I added "from dm.pdb import
>>> zpdb; zpdb.set_trace()" in the doctest ("dm.pdb.zpdb" is my Zope aware
>>> PDB extension)
>>
>> The standard PDB works fine. I'm sure you're extended version can be
>> tweaked to as well.
>
>I have similar issues with ipdb. - similar in sense that I cannot use it
>when running from doctests.
Any any other debugger, too, will have problems.
Usually, I am no fan of "explicit is better than implicit".
But, in this case, I think it would give a much better solution.
Apparently, the debugger integration works by wrapping
some class ("_OutputRedirectingPdb") around a given debugger
and then use this debugger instead of the wrapped on.
How about documenting this wrapping class.
Then any debugger author can use the class to wrap his
debugger and use it.
For this special case, we do not need monkey patching.
import myWrappedDebugger; myWrappedDebugger.set_trace()
(or for the standard debugger:
"from zope.testing import pdb; pdb.set_trace()")
is only slightly more complex than
import pdb; pdb.set_trace()
--
Dieter
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: What happened to the infrae.subversion and py eggs?
Previously Reinout van Rees wrote: > Wichert Akkerman schreef: > >Previously Reinout van Rees wrote: > >>Tres Seaver schreef: > >>>Daniel Nouri wrote: > 1.0dev-r27844 seems to be gone from PyPI. > >>>Such a version should *never* have been "released" to PyPI (any egg / > >>>source dist with an SVN revision number in its filename is *not* > >>>suitable for sharing with the wider world). > >>I've made it a habit *not* to enable the option in setup.cfg that adds > >>those svn revision numbers to the version. It is enabled by default in > >>all/most of the ZopeSkel-generated packages that I use. > > > >I recently started adding that option again for some of my projects. For > >a few frequently updated sites I do a lot of updates & installs without > >making full releases and it is extremely convenient to be able to see > >which exact revision number I'm currently using. > > Those are probably private and not on pypi, I guess? Or do you do that > for a few pypi'ed packages, too? I basically don't trust myself enough > to enable it for public packages :-) I do that frequently for packages which are also on pypi. We do it for all plone.* and plone.app.* packages for example. I just don't upload those dev revisions to pypi. Wichert. -- Wichert Akkerman <[EMAIL PROTECTED]>It is simple to make things. http://www.wiggy.net/ It is hard to make things simple. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: What happened to the infrae.subversion and py eggs?
Wichert Akkerman schreef: Previously Reinout van Rees wrote: Tres Seaver schreef: Daniel Nouri wrote: 1.0dev-r27844 seems to be gone from PyPI. Such a version should *never* have been "released" to PyPI (any egg / source dist with an SVN revision number in its filename is *not* suitable for sharing with the wider world). I've made it a habit *not* to enable the option in setup.cfg that adds those svn revision numbers to the version. It is enabled by default in all/most of the ZopeSkel-generated packages that I use. I recently started adding that option again for some of my projects. For a few frequently updated sites I do a lot of updates & installs without making full releases and it is extremely convenient to be able to see which exact revision number I'm currently using. Those are probably private and not on pypi, I guess? Or do you do that for a few pypi'ed packages, too? I basically don't trust myself enough to enable it for public packages :-) Reinout -- Reinout van Rees Blog: http://vanrees.org/weblog/ [EMAIL PROTECTED] Work: http://zestsoftware.nl/ http://vanrees.org Video: http://reinout.blip.tv/ ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: straighting out the SQLAlchemy integration mess
On 11.04.2008, at 10:09, Brian Sutherland wrote: Unfortunately, it doesn't seem an option, right now, to avoid using ZCML to setup database connections. So I'm not sure how you want to implement your ideas? +1 k, i didn't read the whole thread, but we (lovely systems) use zope.storm but i think its the same story somehow and we do all instance specific configuration in zcml - also database uris. we have a base.zcml config for all instances and include this it and do all instance specific configuration in an instance specific zcml file. this file is then used with the site-definition otpion in zope.conf. i can see no reason why we should avoid zcml except in environments where there is no zcml at all. although it is a few lines of code to get db uris from somewhere else, and this could also be application code regards, bernd smime.p7s Description: S/MIME cryptographic signature ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: straighting out the SQLAlchemy integration mess
Brian Sutherland schreef: Unfortunately, it doesn't seem an option, right now, to avoid using ZCML to setup database connections. So I'm not sure how you want to implement your ideas? Something we did for an LDAP connection that needed the same kind of server-dependent config: put an ldapconfig.py file in zope's etc/ dir and load that if available from the code that configures ldap. Not the best of solutions, probably, but it worked. What I wanted to say: this *is* a way to avoid using zcml! Reinout -- Reinout van Rees Blog: http://vanrees.org/weblog/ [EMAIL PROTECTED] Work: http://zestsoftware.nl/ http://vanrees.org Video: http://reinout.blip.tv/ ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: straighting out the SQLAlchemy integration mess
Andreas Jung wrote: [snip] Possibly a sane and clean solution but way of over-designed. I am interested in simple and robust solutions that work and not in a solution that adds more and more unnecessary layers just the sake of having them :-) It's a 1 liner to set up the connection in this story if you want to hard-code it in Python, and by implementing it as a property, you can get the connection string from any place you want, including environment variables or whatever else your Python code can get to. This cuts the whole debate about where the connection string should come from short, as you can code whatever you want yourself. This has an example of the utility setup: http://faassen.n--tree.net/blog/view/weblog/2008/04/08/0 Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: straighting out the SQLAlchemy integration mess
Brian Sutherland wrote: [snip] Unfortunately, it doesn't seem an option, right now, to avoid using ZCML to setup database connections. So I'm not sure how you want to implement your ideas? collective.lead can do it without ZCML; you set up the right connection by implementing a 'url' property on a (global) utility. This to me seems to be flexible, as the code could get the connection from anywhere. Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: straighting out the SQLAlchemy integration mess
Andreas Jung zopyx.com> writes: > > The way I use collective.lead in my book is to have it look up the > > database settings in a local utility. That utility is editable via a > > control panel page in Plone. I suspect that it'd be quite easy to do > > something similar where the settings were deferred to a global utility > > configured with ZCML or looked up in a config file of some sort. > > > > Once the config is available, it's all about calling getUtility() to get > > hold of a database connection/session. > > Possibly a sane and clean solution but way of over-designed. I am > interested in simple and robust solutions that work and not in a solution > that adds more and more unnecessary layers just the sake of having them collective.lead doesn't enforce this style at all. If you want TTW configurable connections, though, it's a good pattern to use. All collective.lead does is: - Set up database connections and engines properly - Hook into transaction machinery so you don't need to do your own transaction management - Let you look up a database session/connection as a utility You need to register that utility, of course. Lead has a base class that means you just fill in a few template methods (database url, and some callbacks for configuring tables and orm classes). If you want some kind of config-file based or ZCML based version, you'd just construct an appropriate class and register it as a utility on startup. Lead doesn't want to enforce one particular connection parameter configuration model or config file format. That should be the job of a higher level package that is concerned only with getting these parameters, not with the database setup itself. However, you probably still need some imperative code to set up the tables and mappers. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: straighting out the SQLAlchemy integration mess
--On 11. April 2008 08:36:43 +0100 Martin Aspeli <[EMAIL PROTECTED]> wrote: Martijn Faassen wrote: Andreas Jung wrote: --On 10. April 2008 19:10:49 +0200 Brian Sutherland <[EMAIL PROTECTED]> wrote: Just adding my few requirements: - Integration into the component architecture in such a way that I can specify the db connection parameters in ZCML and that database reflection still works. I want to be able to get a handle on SQLAlchemy things by calling getUtility. Please *NO* database specific configurations within ZCML. We're running applications in up three or four different environments and I don't want to maintain instance specific configurations within ZCML. Either we pass such informations through environment variables or the database connections parameters are read from a configuration within the instance configuration. As long as it can also be specified in the code, that way people can build UIs to set up the connection. Perhaps that's what you mean by a "configuration within the instance configuration". The way I use collective.lead in my book is to have it look up the database settings in a local utility. That utility is editable via a control panel page in Plone. I suspect that it'd be quite easy to do something similar where the settings were deferred to a global utility configured with ZCML or looked up in a config file of some sort. Once the config is available, it's all about calling getUtility() to get hold of a database connection/session. Possibly a sane and clean solution but way of over-designed. I am interested in simple and robust solutions that work and not in a solution that adds more and more unnecessary layers just the sake of having them :-) Andreas pgpZS98rSgYnq.pgp Description: PGP signature ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: straighting out the SQLAlchemy integration mess
--On 11. April 2008 10:09:16 +0200 Brian Sutherland <[EMAIL PROTECTED]> wrote: On Fri, Apr 11, 2008 at 05:07:56AM +0200, Andreas Jung wrote: --On 10. April 2008 21:29:43 +0200 Brian Sutherland <[EMAIL PROTECTED]> wrote: What defines if something is the instance configuration? Isn't ZCML just another format of configuration file? I agree it's evil to hardcode database connections in the application as they must be under the control of the sysadmin. But that doesn't preclude them from being expressed in zcml. The point is: we don't need yet another configuration language for the configuration of a site. ZCML should be used for the purpose it has been intended for: for the configuration of the components of your site (easily spoken). This kind of configuration is static if you setup a site for project X. But settings like ports, paths etc. often differ between dev, test and production environments. We already have ZConfig as standard cofniguration language for instance specific configuration aspects. In addition: we often do use environment variables and .ini files for additional configurations. Introducing another configuration language is stupid. Especially because system administrators or co-workers know little or nothing about ZCML. This is an unnecessary introduction of complexity. Setting an environment variable with a DSN is much easier then having to fiddle around with ZCML and XML...totally unnecessary overheadkeep it simple ...using ZCML in this configuration context is only for XML masochists. Unfortunately, it doesn't seem an option, right now, to avoid using ZCML to setup database connections. So I'm not sure how you want to implement your ideas? I have no need for ideas to be implemented. We're using SA within in several large projects based on z3c.sqlalchemy. And nothing beats the simplicity of of an .ini file or an environment variable. In general, I think if you want your configuration expressed in ZConfig and available as a utility, you have a problem. I'm not sure of: * How do I register a utility in ZConfig? z3c.sqlalchemy provides a method for registering a SA connection as a named utility. This code is usually called during the startup phase of Zope within the related initialize() methds of our products. The DSN are acquired from well-known environment variables...this just works - easy, simple, reliable. * How will the above registration work properly in functional tests (especially with tearing down and setting up layers)? Functional tests in our company are tests done by humans. Why? Our applications are very complex with lots of dependencies. Not speaking of lots of involved databases, big storages.. Andreas pgp98vhOQjdEm.pgp Description: PGP signature ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Zope Tests: 5 OK
Summary of messages to the zope-tests list. Period Thu Apr 10 11:00:00 2008 UTC to Fri Apr 11 11:00:00 2008 UTC. There were 5 messages: 5 from Zope Tests. Tests passed OK --- Subject: OK : Zope-2.8 Python-2.3.6 : Linux From: Zope Tests Date: Thu Apr 10 20:57:43 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-April/009385.html Subject: OK : Zope-2.9 Python-2.4.4 : Linux From: Zope Tests Date: Thu Apr 10 20:59:14 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-April/009386.html Subject: OK : Zope-2.10 Python-2.4.4 : Linux From: Zope Tests Date: Thu Apr 10 21:00:44 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-April/009387.html Subject: OK : Zope-2.11 Python-2.4.4 : Linux From: Zope Tests Date: Thu Apr 10 21:02:14 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-April/009388.html Subject: OK : Zope-trunk Python-2.4.4 : Linux From: Zope Tests Date: Thu Apr 10 21:03:44 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-April/009389.html ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: straighting out the SQLAlchemy integration mess
On Fri, Apr 11, 2008 at 05:07:56AM +0200, Andreas Jung wrote: > --On 10. April 2008 21:29:43 +0200 Brian Sutherland > <[EMAIL PROTECTED]> wrote: >> What defines if something is the instance configuration? >> >> Isn't ZCML just another format of configuration file? I agree it's evil >> to hardcode database connections in the application as they must be >> under the control of the sysadmin. But that doesn't preclude them from >> being expressed in zcml. > > The point is: we don't need yet another configuration language for the > configuration of a site. ZCML should be used for the purpose it has been > intended for: for the configuration of the components of your site (easily > spoken). This kind of configuration is static if you setup a site for > project X. But settings like ports, paths etc. often differ between dev, > test and production environments. > > We already have ZConfig as standard cofniguration language for instance > specific configuration aspects. In addition: we often do use environment > variables and .ini files for additional configurations. Introducing another > configuration language is stupid. Especially because system administrators > or co-workers know little or nothing about ZCML. This is an unnecessary > introduction of complexity. Setting an environment variable with a DSN is > much easier then having to fiddle around with ZCML and XML...totally > unnecessary overheadkeep it simple ...using ZCML in this configuration > context is only for XML masochists. Unfortunately, it doesn't seem an option, right now, to avoid using ZCML to setup database connections. So I'm not sure how you want to implement your ideas? In general, I think if you want your configuration expressed in ZConfig and available as a utility, you have a problem. I'm not sure of: * How do I register a utility in ZConfig? * How will the above registration work properly in functional tests (especially with tearing down and setting up layers)? -- Brian Sutherland ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: straighting out the SQLAlchemy integration mess
Martijn Faassen wrote: Andreas Jung wrote: --On 10. April 2008 19:10:49 +0200 Brian Sutherland <[EMAIL PROTECTED]> wrote: Just adding my few requirements: - Integration into the component architecture in such a way that I can specify the db connection parameters in ZCML and that database reflection still works. I want to be able to get a handle on SQLAlchemy things by calling getUtility. Please *NO* database specific configurations within ZCML. We're running applications in up three or four different environments and I don't want to maintain instance specific configurations within ZCML. Either we pass such informations through environment variables or the database connections parameters are read from a configuration within the instance configuration. As long as it can also be specified in the code, that way people can build UIs to set up the connection. Perhaps that's what you mean by a "configuration within the instance configuration". The way I use collective.lead in my book is to have it look up the database settings in a local utility. That utility is editable via a control panel page in Plone. I suspect that it'd be quite easy to do something similar where the settings were deferred to a global utility configured with ZCML or looked up in a config file of some sort. Once the config is available, it's all about calling getUtility() to get hold of a database connection/session. Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [zope.testing.doctest] a nasty surprise
On Thu, 10 Apr 2008 11:50:35 -0400, Benji York wrote:
> Dieter Maurer wrote:
>> As the community (apparently) strongly favors "doctest" over
>> "unittest", I wrote my first test based on "zope.testing.doctest". And
>> promptly, I was badly surprised
>>
>> In order to analyse a difficult problem, I added "from dm.pdb import
>> zpdb; zpdb.set_trace()" in the doctest ("dm.pdb.zpdb" is my Zope aware
>> PDB extension)
>
> The standard PDB works fine. I'm sure you're extended version can be
> tweaked to as well.
I have similar issues with ipdb. - similar in sense that I cannot use it
when running from doctests.
ipdb is a lightweight component that enables entering ipython when
"import ipdb; ipdb.set_trace()" is called. Those of you that use ipython
and also develop with doctests a lot, will understand why it would be a
nice to have also from doctests.
In case of ipdb I had no parameters problem like Dieter encountered with
zpdb, however I observed that the doctest runner needs to redirect input
and output while running tests, which are specifically routed back to
stdin and stdout while pdb.set_trace is entered, to enable interactivity.
This is done by monkeypatching pdb if I remember well. But the same does
not happen with ipdb, of course.
So, my conclusion is that while doctests work well with the standard pdb,
any alternate debugger must be supported explicitely (by changing the
doctest code). As with ipdb, this may be the case with zpdb as well.
Best wishes
--
Balazs Ree
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
