[Zope-dev] Re: [Zope2] ZODB load optimization ineffective due to ExtensionClass flaw
Dieter Maurer wrote: The ZODB allows to include the class (a reference to it) into persistent references in order to avoid loading the object state when a ghost object needs to be created for the persistent reference. For this to work, the class must not require arguments to "__new__". The ZODB checks whether the class defines a "__getnewargs__" method. It it does, the ZODB concludes that its "__new__" might need arguments and does not try to optimize. Unfortunately, "ExtensionClass.Base" defines "__getnewargs__". It is the base class for all Zope2 classes. Zope2 persistent references are not optimized... An analysis of our application revealed that the missed optimization opportunity increases the amount of storage loads by a factor of 4 to 5. Storage loads are often the dominating factor -- especially for searches. @Jim: Is the "__getnewargs__" definition in "ExtensionClass.Base" vital? It returns the empty tuple. Thus, there seems to be a chance to get rid of it. I know that some code (especially in "ExtensionClass/picke/pickle.c") needs to be changed as it depends on the existence of "__getnewargs__". If you think, there is a chance to get rid of "__getnewargs__", I can do the work and put the changes into the collector. Zope2 might become a bit faster... I'd have to review this, but I'm pretty sure that we dealt with this issue, but I could be mistaken. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: http access to svn repos?
Mark Hammond wrote: Chris quoting Jim: ... whether SVN or not, I'm guessing any use of HTTP basic authentication mechanism qualifies as "extremely lame"! I've no idea if this is what Jim meant though :) OK, for those not familiar with svn/HTTP authentication, as I understand it you have to authenticate for each session and your credentials are cached in clear text in your home directory. The storage of clear-text credentials is obviously lame, as is the necessity to provide then for each svn session. With the current ssh-based mechanism, I authenicate once when I log into my machine and don't have to authenticate again for the remainder of that OS session, during which I can log into many remote machines and access many different Subversion and CVS repositories without having to reenter credentials. I find this to be a major convenience. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] [Zope2] ZODB load optimization ineffective due to ExtensionClass flaw
The ZODB allows to include the class (a reference to it) into persistent references in order to avoid loading the object state when a ghost object needs to be created for the persistent reference. For this to work, the class must not require arguments to "__new__". The ZODB checks whether the class defines a "__getnewargs__" method. It it does, the ZODB concludes that its "__new__" might need arguments and does not try to optimize. Unfortunately, "ExtensionClass.Base" defines "__getnewargs__". It is the base class for all Zope2 classes. Zope2 persistent references are not optimized... An analysis of our application revealed that the missed optimization opportunity increases the amount of storage loads by a factor of 4 to 5. Storage loads are often the dominating factor -- especially for searches. @Jim: Is the "__getnewargs__" definition in "ExtensionClass.Base" vital? It returns the empty tuple. Thus, there seems to be a chance to get rid of it. I know that some code (especially in "ExtensionClass/picke/pickle.c") needs to be changed as it depends on the existence of "__getnewargs__". If you think, there is a chance to get rid of "__getnewargs__", I can do the work and put the changes into the collector. Zope2 might become a bit faster... -- Dieter ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: http access to svn repos?
Mark Hammond schrieb: Chris quoting Jim: ... I would support HTTP anonymous checkouts. I'm really against writable HTTP checkouts because I consider the credentials mechanism for HTTP access to be extremely lame. whether SVN or not, I'm guessing any use of HTTP basic authentication mechanism qualifies as "extremely lame"! I've no idea if this is what Jim meant though :) Well, I hope ;) he meant client certificates. This is doable but a bit of work for the certificate people to issue one to the user in addition to the ssh-pubkey stuff. Not actually quite in line w/ what you should do as a CA but possible and not more insecure then current ssh-pubkey auth would be a script which can be run with the ssh-useraccount and produces/registeres a given client certificate for that user. Something like: ssh cert.zope.org generate >mycert.csr when your ssh-pubkey is set up. And likewise ssh cert.zope.org retract http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
RE: [Zope-dev] Re: http access to svn repos?
Chris quoting Jim: > > lame program for uploading keys, I find the ssh-based access > > mechanism to be far more usable and secure. > > Secure, maybe, but is it really worth it? > > Usable? Don't agree, especially if you're trying to develop on Windows... As a data-point, for the last year or 2, I've had good success with putty's ssh2 keys on Windows. And quoting out-of-order, Jim wrote: > I would support HTTP anonymous checkouts. I'm really against > writable HTTP checkouts because I consider the credentials > mechanism for HTTP access to be extremely lame. whether SVN or not, I'm guessing any use of HTTP basic authentication mechanism qualifies as "extremely lame"! I've no idea if this is what Jim meant though :) Cheers, Mark ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: http access to svn repos?
Jim Fulton wrote: At one point, enabling the 'http:' checkout gateway was a sure-fire recipe for getting SVN's knickers in a twist, which is why we disabled it. Or maybe that was ViewCSV. Actually, it was BekeleyDB. :) And Jens has fixed that now, iirc? The main obstical was that it required apache 2 and, at the time we were running apache 1 and I didn't want to spend the time figuring out the apache access. Is cvs.zope.org still running Apache 1? I would support HTTP anonymous checkouts. I'm really against writable HTTP checkouts because I consider the credentials mechanism for HTTP access to be extremely lame. Can you elaborate a little? I'm guessing you may mean that the subversion client has a propensity for storing cleartext passwords but is there anything else? lame program for uploading keys, I find the ssh-based access mechanism to be far more usable and secure. Secure, maybe, but is it really worth it? Usable? Don't agree, especially if you're trying to develop on Windows... cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] http access to svn repos?
Jim Fulton wrote: Not in principle. In fact, I would find it convenient for a particular project. I doubt that anyone with access to that machine has time, \ although I can only speak for myself. If I could get access, I'd make time ;-) I think I used to have it, and I may well still do, but I've forgotten what user I should "sudo su -" to in order to be able to do stuff with Apache... What should I be trying? Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
