[Zope] Resolving ConflictError
I've recently encountered my first ConflictError. I am hosting multiple sites in this ZODB, and each of them is using a LDAPLoginAdapter as an acl_users: Traceback (innermost last): Module Zope2.App.startup, line 167, in zpublisher_exception_hook/li Module ZPublisher.Publish, line 120, in publish/li Module Zope2.App.startup, line 233, in commit/li Module transaction._manager, line 84, in commit/li Module transaction._transaction, line 381, in commit/li Module transaction._transaction, line 379, in commit/li Module transaction._transaction, line 424, in _commitResources/li Module ZODB.Connection, line 462, in commit/li Module ZODB.Connection, line 495, in _commit/li ConflictError: database conflict error (oid 0x013b3f, class Products.LDAPLoginAdapter.LDAPLoginAdapter.LDAPLoginAdapter) Once this error started occurring on one of the acl_users, I've now seen it on several of them. The websites are still useable, as this error only occurs several times per day, out of ~100s of requests. Searching on google, however, doesn't really give me too much insight into figuring out what my next step is. I'd obviously like to figure out what code is causing the ConflictError, and how I can get rid of it and/or fix the code that is causing it. I've read enough from google searching to familiarize myself with the fs* tools (fsrefs, fsdump), but I'm not sure where to go from there. Thanks in advance, John Ziniti ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Zope-2.8.1, LocalFS, and the importance of _p_jar
I've encountered the issue described here: http://mail.zope.org/pipermail/zope/2005-August/161120.html on a recently upgraded Zope using LocalFS. I have tracked down the issue to the fact that the object that LocalFS hands to ZODB is a WrapperObject, and this object does not have an oid. This raises an AssertionError during the _commit method of ZODB.Connection. I have traced the code a little bit and I see that this block of code will be avoided for these objects if the objects do not appear in self._registered_objects, which I can achieve by making sure that the object has no _p_jar attribute. If I comment out the line in the LocalFS object where the wrapper object's _p_jar is set to the parent object's _p_jar, everything works fine, with no (apparent) problems. [ I tried also setting obj._p_oid = self._p_oid, but then Zope has pickling problems trying to pickle instances of the wrapper class ]. The problem I have with my solution is that I have no clue what _p_jar is. What is _p_jar used for within the Zope model? What are the implications of having an object with self._p_jar=None? What are the implications of a transient object (not stored in the ZODB but on the local filesystem), with self._p_jar=None? Thanks in advance for any information, John Ziniti ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Hard-coded Content-type:text/xml as xmlrpc in Zope (2.8.1)
Back in June 2004, I had the following problem: http://mail.zope.org/pipermail/zope/2004-June/151436.html http://mail.zope.org/pipermail/zope/2004-June/151492.html http://mail.zope.org/pipermail/zope/2004-June/151497.html http://mail.zope.org/pipermail/zope/2004-June/151498.html which came about because HTTPRequest.py was hard-coded to assume that any request with a Content-type of text/xml is an xmplrpc call and handles the request as such. I resolved this issue with the help of [EMAIL PROTECTED], who gave me the XmlFix product, which monkey-patches the HTTPRequest.processInputs method to respond a bit more appropriately. We noted in the thread that although Zope's behavior is broken, it is unlikely to change for historical reasons. I am fine with this and felt comfortable with the simplicity and supportability of the monkey-patch. This monkey-patch was no longer effective recently when I upgraded our Zope instances to 2.8.1. Zope-2.8.1 now additionally has the zope.app.publication. HTTPPublicationRequestFactory class, which also assumes that text/xml means xmlrpc (in fact, it assumes that anything that startswith('text/xml') is an xmlrpc call). Not fixing old code which acts in strange hard-coded ways is something I understand, but adding new code that also acts that way seems like it is getting further away from good :) Can anything be done to change the behavior in the HTTPPublicationRequestFactory so that it is not so obtuse? My monkey-patch now has to munge two methods to make xml uploads work and I am less happy with it than I was before. Thanks for your time, John Ziniti ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] LDAPUserFolder at Zope root
Oscar Figueiredo wrote: I tried the dummy way : delete the acl_users with an intent to instanciate a LDAPUSerFolder afterwards but that locked me out in a way I could do nothing You need to log in as the emergency user, delete the old acl-users, instantiate the new LDAPUserFolder, and then logout and log in again as someone from the LDAPUserFolder who has Manager privileges (just make sure that such a user exists!). If any problems ever arise with the LDAPUserFolder, you'll need to log in as the emergency user in order to fix them. HTH, JZ ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Session Variable Error
Asad Habib wrote: Actually, it does exist as I mentioned in my original message. If it did not exist, then dtml-var REQUEST.SESSION would yield a value of None for it. Also, if this were the case, then simply changing the name of the variable would not solve the problem. I think that Andreas is pointing out an inconsistency in your original post, and you should check and make sure that the inconsistency does not actually exist in your program. First you indicate that you set a variable called undergradProjectsDirected: I declare using the following syntax: dtml-call REQUEST.SESSION.set('undergradProjectsDirected', [ ]) But then you indicate that you are using the following to add items: I am using the following statement to append items: dtml-call (REQUEST.SESSION.get('undergraduateProjectsDirected')).append([student_n ame, description]) Notice that this time you are using undergraduateProjectsDirected. Please check and then indicate whether this is true in your actual program. Obviously, if you are goign to change the name of the variable, you have to be consistent. Andreas is just indicating that perhaps you haven't been careful to ensure this. HTH, JZ ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope-dev] Re: SOAP Support for ZOPE
Brian Lloyd wrote: - It is a pain to do anything with SOAP because the publisher has a hard-coded idea that anything xml must be xml-rpc This would be minimally disruptive to the Zope core, while enabling people interested in SOAP to evolve different solutions without everybody having to buy into a particular approach or implementation right now. +1 I think these ideas are great, but wasn't it also you who said, Unfortunately, though, this misfeature has been around for a long time and would break a lot of people if the default were changed. I think getting this changed might be met with some amount of resistance. If we're voting ;-) I vote for biting the bullet and making the change today. John Ziniti ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] SOAP Support for ZOPE
Aruna Kathiria wrote: I did some work regarding SOAP support on ZOPE and published this document on zope.org. The link for this document is http://zope.org/Members/arunacgx/SOAP%20Support%20on%20Zope/file_view I would like to get feedback/suggestion regarding this document. One preliminary suggestion: In the section entitled 3.2.4 Correcting HTTPRequest.py, I would suggest that instead of having users edit the file, you could instead distribute your product as a Monkey Patch Zope Product that overrides the HTTPRequest.processInputs method with your altered version. You can see an example of this in the XmlFix Product attached to the followign mailing-list item: http://mail.zope.org/pipermail/zope/2004-June/151497.html Basically you would do this: code ### Import the HTTPRequest object we want to alter from ZPublisher.HTTPRequest import HTTPRequest ### Save the *old* processInputs method def initialize(context): HTTPRequest._realProcessInputs = HTTPRequest.processInputs HTTPRequest.processInputs = processInputs ### Write a new processInputs method that does what we want def processInputs(self, *args, **kw): if myProductShouldHandleThisRequest: do_stuff() else: ### Hand over to the original processInputs return self._realProcessInputs(*args, **kw) /code Since this is a Zope Product, the above code would run once every time you started Zope, and is much more portable between Zope installations than having the admin edit code. Hope that helps, John Ziniti ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope 2.7.0 and Pythom Image Library
Tor Oskar Wilhelmsen wrote: I'm trying to get the PIL(last version on python 2.3) working with Zope, but it seems that it's not working. I use the code: file = StringIO.StringIO(data) im = Image.open(file) im.show() where data is datafrom a blob in a db. This code is working outside Zope, but at once when i try to make it a product in Zope, I get this error: IOError: cannot identify image file I've had problems using PIL-based products in Zope because Zope has its own module called ImageFile and Zope does some sys.path munging so that when certain PIL modules do import ImageFile, they import the Zope version, instead of the ImageFile.py that comes with PIL. This causes lots of hard-to-track errors that have weird error messages ( I always get BadFont ). I have fixed this problem before by editing the module for the type of image I am using to do a more explicit import. For example, for JPEG images, I edit PIL/JpegImagePlugin.py to read: from PIL import ImageFile This is at or around line 34, which reads import ImageFile. So, you could make the changes to the files corresponding to the Image type you are using, then *restart Zope* (the fix is in site-packages so a Refresh won't do it!) and then give it a try. I'd be happy to hear if this helps you, John Ziniti ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope products and folders
Stephen Rudd BIOINF wrote: before releasing. Could someone explain (or point me to resources) how to turn a product into a folder like object so that I can bind images, DTML or python scripts directly into the product - I am stumped and cannot find any explanation Your first step will be to have your Folder-like object inherit from ObjectManager. from OFS.ObjectManager import ObjectManager class FolderLike(ObjectManager, Implicit, Item): manage_options = ( ObjectManager.manage_options + Item.manage_options ) That should get you started. For more questions like this, I would post to plain ol' zope list ([EMAIL PROTECTED]). I lurk that list and have written plenty of Folder-like products, and I'd be happy to help in any way I can. JZ ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Prevayler and Object Prevalence
Any ZODB developers read this article at /. and the accompanying one at developerWorks? Would anyone care to comment about the relevance/comparisons of prevalence to the concepts of persistence used in ZODB and/or ZEO? http://developers.slashdot.org/article.pl?sid=03/03/03/1220222mode=nestedtid=156threshold=1 Just looking for some info. Thanks in advance, JZ ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] from ImageFile import ImageFile
__The short story__: I would like to recommend to all Product developers that for Product initialization, they please the statement: from App.ImageFile import ImageFile instead of: from ImageFile import ImageFile __The long story__: A colleague of mine recently went through much trouble, which eventually boiled down to a name- clash with piddle/PIL. Zope imports the name ImageFile into it's global namespace during startup. A lot of Products take advantage of this during Product initialization, by importing the name using: form ImageFile import ImageFile The Zope source, though, states explicitly that this module will disappear in the future. At the very least, the following Products have trouble (and I suspect many more, as well): TinyTablePlus MetaPublisher LDAPAdapter ZDconfera ZSyncer ZOracleDA NISUserFolder Thanks, Ziniti ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] improved logging
http://freshmeat.net/projects/log4py/ Romain Slootmaekers wrote: Yo, questions wrt logging: -) What are the plans to improve the rather primitive logging facilities in zope ? I couldn't find anything in the plan for 2.6, nor on the 3.0 homepage. I think we need at least an extendibl framework with following types of components: - Loggers (fi, file logger, db loggers, remote logger, nulllogger,...) - Filters (fi on severity, on subsystem, ...) - LogBus (central accesspoint for the logging framework, where you can (dynamically) hook your loggers, filters etc... if time is an issue, we could just copy the design of fi Log4J which is a good logging framework for apache. -) If there are no such plans, can we hack something up ourselves and add/donate it to the product ? I hate writing stuff that gets replaced by something else afterwards input appreciated. Sloot. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope 2.6 planning - call for contributors!
Let's get a discussion started to define 2.6 I'd like to see the ZSyncer Product, or a variant thereof, included in Zope by default. That is, I'd like Synchronization, to a be a default property of Zope objects, so that objects/content can be pushed and pulled between two Zope installations. I venture a guess that the development/production model is common, and this helps quite a bit in maintaining this model. Ziniti ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] _getProductRegistryData('zclasses') returns non-existent Products -- permission to muck around?
My ProductRegistry has references to products which no longer exist in my Zope installation. This causes a variety of problems in lots of different places when other products try to get more information about all the products in the ProductRegistry (obviously). 1) How does this occur? 2) Is it OK to put my boots on, get in there and manually delete entries which refer to non-existent Products? TIA Ziniti ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] ghosts of removed products haunting ZSQL Methods?
I'm ready and eager to blame Z classes for everything :) Yeah, me too, hatin' em right about now ... % cd lib/python % python2.1.1 import Zope app = Zope.app() app.fixupZClassDependencies(rebuild=1) get_transaction().commit() Hopefully that will do it for you. Found that on a mailinglist archive last night. Tried it but it doesn't work. The offending Product's ZClasses are still in the Registry (but nowhere else). ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] ghosts of removed products haunting ZSQL Methods?
Dieter Maurer wrote: John Ziniti writes: I used to have the Product TrackerBase installed and have removed it. Now, however, attempts to access the Advanced tab of ZSQL Methods is broken because a chain of events attempt to get info about a the non-existent Product: ... 2) Is this schizophrenic state my fault? Probably not. I didn't think so, but I would think that the code which deletes Products and their associated ZClasses would be pretty rock solid at this point ?? Looks like a bug, you should report to http://collector.zope.org;. I would, but as a developer myself, a bug that has a symptom bu no real evidence of a cause is one I wouldn't really think to look at it. My ZODB is broken is basically the best I can come up with at this point as a way to explain my problem. I can talk all the way in to Zope.app()._getProductRegistryData() about the way the problem manifests itself, but it happened at some point over the last ... oh ... three months of development??? It'll get ignored, for sure ... ( no offense intended) 3) How to rectify the problem (Make *all* of Zope know that TrackerBase is gone)? I think this is the right approach. Yep. I really just want to try something akin to: old_registry=app._getProductRegistryData('zclasses') app._SetProductRegistryData('zclasses')=old_registry[1:] ... but I'd like to hear from the guy who wrote _getProductRegistryData first before I even try it on a backup-devlopment-Zope. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] ghosts of removed products haunting ZSQL Methods?
I used to have the Product TrackerBase installed and have removed it. Now, however, attempts to access the Advanced tab of ZSQL Methods is broken because a chain of events attempt to get info about a the non-existent Product: lib/python/Shared/DC/ZRDB/dtml/advanced.dtml calls manage_product_zclass_info ../ZRDB/DA.py.manage_product_zclass_info (line 546) calls self.aq_acquire('_getProductRegistryData')('zclasses') One of these items is the dict: {'id': 'Tracker', 'meta_type': 'Tracker', 'product': 'Tracker', 'meta_class': ZClass instance at d55af0} Then, line 548 calls if hasattr(z._zclass_,'_p_deactivate'). The attempt to access z._zclass_ fails because this product doesn't exist. 1) the call to z._zclass_ doesn't actuallt do anything except: ## Eek, persistent 2) Is this schizophrenic state my fault? 3) How to rectify the problem (Make *all* of Zope know that TrackerBase is gone)? Thanks in advance, traceback follows, Ziniti Error Type: ImportError Error Value: No module named TrackerBase Troubleshooting Suggestions * The URL may be incorrect. * The parameters passed to this resource may be incorrect. * A resource that this resource relies on may be encountering an error. For more detailed information about the error, please refer to the HTML source for this page. If the error persists please contact the site maintainer. Thank you for your patience. User is Last modified 2002/01/07 16:37:34.6452 GMT+0 © Copyright 2000 Channing Lab Questions or comments about this site? All Rights Reserved. Traceback (innermost last): File /u05/ilocal/opt/Zope-2.4.0-src/lib/python/ZPublisher/Publish.py, line 223, in publish_module File /u05/ilocal/opt/Zope-2.4.0-src/lib/python/ZPublisher/Publish.py, line 187, in publish File /u05/ilocal/opt/Zope-2.4.0-src/lib/python/Zope/__init__.py, line 226, in zpublisher_exception_hook (Object: new_sql_2) File /u05/ilocal/opt/Zope-2.4.0-src/lib/python/ZPublisher/Publish.py, line 171, in publish File /u05/ilocal/opt/Zope-2.4.0-src/lib/python/ZPublisher/mapply.py, line 160, in mapply (Object: manage_advancedForm) File /u05/ilocal/opt/Zope-2.4.0-src/lib/python/ZPublisher/Publish.py, line 112, in call_object (Object: manage_advancedForm) File /u05/ilocal/opt/Zope-2.4.0-src/lib/python/Shared/DC/Scripts/Bindings.py, line 324, in __call__ (Object: manage_advancedForm) File /u05/ilocal/opt/Zope-2.4.0-src/lib/python/Shared/DC/Scripts/Bindings.py, line 354, in _bindAndExec (Object: manage_advancedForm) File /u05/ilocal/opt/Zope-2.4.0-src/lib/python/App/special_dtml.py, line 241, in _exec (Object: manage_advancedForm) File /ilocal/opt/Zope-2.4.0-src/lib/python/Shared/DC/ZRDB/DA.py, line 549, in manage_product_zclass_info (Object: new_sql_2) File /u05/ilocal/opt/Zope-2.4.0-src/lib/python/ZODB/Connection.py, line 544, in setstate ImportError: (see above) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] security.declareProtected doesn't always work?
The basic security mechanism uses the attribute m__roles__ in order to protect m. If this attribute it None, then m is public. Otherwise, it is expected to be a sequence of roles that are allowed to use m. But, ExtensionsClass brings with it computed attributes. This allows m__roles__ to be not a sequence but a method returning a sequence. When you protect m with a permission p, then m__roles__ is set to PermissionRole(p). This instance dynamically evaluates into a sequence of roles by crawling up the aq_container (which is correctly aq_parent after aq_inner) chain and translating p into roles by interpreting the permission-to-role mapping it finds on its way to the application object. Therefore, declarePublic works for non-wrapped instances while declareProtected requires the wrapping. Dieter Very well put, Dieter -- both as an explanation of the problem at hand, as well as a general description of one of the more esoteric regions of Zope. I think it belongs in a document somewhere. Thanks. I knew I read these email lists for a reason. Ziniti ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: EINTR ... was Re: [Zope-dev] browser closing connection
Replace your Python 2.1.1 asyncore.py with the one that is attached. I've been using it for months now with no problems. Notice, however, that it doesn't work on WinNT, b/c the author didn't know what EINTR looked like on NT Leonardo Rochael Almeida wrote: So, which is the official way of fixing Zope 2.4.3? wait for a hotfix? apply Matthew's patch? steal asyncore from Python 2.2? On Tue, 2001-12-11 at 12:52, Jeremy Hylton wrote: MTK == Matthew T Kromer [EMAIL PROTECTED] writes: MTK For what its worth, I tracked this down in the sources and MTK confirmed that in Zope 2.3, we shipped a modified asyncore.py MTK with Medusa that handled EINTR, but in Zope 2.4 we used stock MTK Python's asyncore which does NOT handle EINTR being returned MTK from select(). IMHO, the distributed Python 2.1 asyncore MTK behavior is incorrect. This is fixed in Python 2.2. [...] # -*- Mode: Python; tab-width: 4 -*- # $Id: asyncore.py,v 1.1.1.3 2001/02/08 13:08:34 tdickenson Exp $ # Author: Sam Rushing [EMAIL PROTECTED] # == # Copyright 1996 by Sam Rushing # # All Rights Reserved # # Permission to use, copy, modify, and distribute this software and # its documentation for any purpose and without fee is hereby # granted, provided that the above copyright notice appear in all # copies and that both that copyright notice and this permission # notice appear in supporting documentation, and that the name of Sam # Rushing not be used in advertising or publicity pertaining to # distribution of the software without specific, written prior # permission. # # SAM RUSHING DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, # INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN # NO EVENT SHALL SAM RUSHING BE LIABLE FOR ANY SPECIAL, INDIRECT OR # CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS # OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, # NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN # CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. # == import exceptions import select import socket import string import sys import os if os.name == 'nt': EWOULDBLOCK = 10035 EINPROGRESS = 10036 EALREADY= 10037 ECONNRESET = 10054 ENOTCONN= 10057 ESHUTDOWN = 10058 EINTR = 0 # what should this be? else: from errno import EALREADY, EINPROGRESS, EWOULDBLOCK, ECONNRESET from errno import ENOTCONN, ESHUTDOWN, EINTR try: socket_map except NameError: socket_map = {} class ExitNow (exceptions.Exception): pass DEBUG = 0 def poll (timeout=0.0, map=None): global DEBUG if map is None: map = socket_map if map: r = []; w = []; e = [] for fd, obj in map.items(): if obj.readable(): r.append (fd) if obj.writable(): w.append (fd) while 1: try: r,w,e = select.select (r,w,e, timeout) except select.error, v: if v[0] != EINTR: raise else: break if DEBUG: print r,w,e for fd in r: try: obj = map[fd] try: obj.handle_read_event() except ExitNow: raise ExitNow except: obj.handle_error() except KeyError: pass for fd in w: try: obj = map[fd] try: obj.handle_write_event() except ExitNow: raise ExitNow except: obj.handle_error() except KeyError: pass def poll2 (timeout=0.0, map=None): import poll if map is None: map=socket_map # timeout is in milliseconds timeout = int(timeout*1000) if map: l = [] for fd, obj in map.items(): flags = 0 if obj.readable(): flags = poll.POLLIN if obj.writable(): flags = flags | poll.POLLOUT if flags: l.append ((fd, flags)) r = poll.poll (l, timeout) for fd, flags in r: try: obj = map[fd] try: if (flags poll.POLLIN): obj.handle_read_event() if (flags poll.POLLOUT): obj.handle_write_event() except ExitNow: raise ExitNow except: obj.handle_error() except KeyError: pass def loop (timeout=30.0, use_poll=0, map=None): if use_poll: poll_fun = poll2 else: poll_fun = poll if map is None:
Re: [Zope-dev] yes, segv11 and Broken pipes
Wasn't this the problem where asyncore.py is not catching the operating system's EINTR. I used to get these all the time, and was able to stop it using a modified asyncore which loops on an OS select() call, restarting the call if it catches EINTR from the OS ... I can send a modified asyncore.py for anyone who wants to give it a try?? Ziniti Jens Quade wrote: Dirk Datzert [EMAIL PROTECTED] writes: Its a Linux 2.2.19. What does IIRC means ? If I remember correctly. I could fix the problem (or a similar one) last summer by changing the Linux kernel. http://mailman.beehive.de/pipermail/zope/2001-June/000590.html http://mailman.beehive.de/pipermail/zope/2001-November/000923.html ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] getPhysicalPath differs for function vs. __getattr__
Sorry the subject is not so discriptive, but here is the issue I am dealing with. I wonder whether this has to do with the way Zope deals with objects or has something to do with Python: I have an Python Product class Object. With the following bits of code, I get different results from a dtml-var, depending on what I use. class Object(PropertyManager, Implicit, Item): def get_path(self): Return a path to myself path=string.join(self.getPhysicalPath()[1:], '/') return path def __getattr(self, attr): Return a path to myself if attr == 'path': path=string.join(self.getPhysicalPath()[1:], '/') return path else: raise AttributeError, attr Now the following two dtml tags yield: dtml-var get_path() = /path/to/object dtml-var path = object Hopefully you see what I mean. Why does this happen? Can I put it in the collector as a bug? TIA, Ziniti ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] file descriptors on Solaris
I am running into a problem where Zope is trying to open too many file descriptors (256) in order to process a POST. I'm not sure how to phrase this question, but my reading has suggested that this limit may be set by the FILE struct in /usr/incldue/stdio.h. Does anyone know if this is used in Python/Zope? My first impression was that this problem would be easy to solve -- just up the number of FD's, but now, I'm not so sure. Any suggestions? Here's a traceback -- but I don't think it's much help Traceback (innermost last): File /u05/ilocal/opt/Zope-2.4.0-src/lib/python/ZPublisher/Publish.py, line 223, in publish_module File /u05/ilocal/opt/Zope-2.4.0-src/lib/python/ZPublisher/Publish.py, line 187, in publish File /u05/ilocal/opt/Zope-2.4.0-src/lib/python/Zope/__init__.py, line 226, in zpublisher_exception_hook (Object: ApplicationDefaultPermissions) File /u05/ilocal/opt/Zope-2.4.0-src/lib/python/ZPublisher/Publish.py, line 136, in publish File /u05/ilocal/opt/Zope-2.4.0-src/lib/python/ZPublisher/HTTPRequest.py, line 405, in processInputs File /ilocal/lib/python2.1/cgi.py, line 517, in __init__ File /ilocal/lib/python2.1/cgi.py, line 606, in read_multi File /ilocal/lib/python2.1/cgi.py, line 519, in __init__ File /ilocal/lib/python2.1/cgi.py, line 616, in read_single File /ilocal/lib/python2.1/cgi.py, line 636, in read_lines File /ilocal/lib/python2.1/cgi.py, line 723, in make_file File /ilocal/opt/Zope-2.4.0-src/lib/python/tempfile.py, line 155, in TemporaryFile OSError: [Errno 2] No such file or directory ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] file descriptors on Solaris
:( I was hoping against hope that this wouldn't be the answer ... I think that the hard limit onSolaris must be 256, because ulimit -n 200 seems to have the appropriate effect of making Zope complain about too many open files ... but anything higher than 256 gets a File not found ... Let's try this from another angle ... why does Zope need to open so many files just to process a *large-but-not-really-large* POST? Can I change something about the way the form is set up (I use a lot of :records) to circumvent the problem? Andreas Jung wrote: There is always a hard limit of filedescriptors (based on the kernel configuration). But usually you can change the number of descriptors using limit descriptors under tcsh/csh. Under Bash I think you must call ulimit. Andreas - Original Message - From: John Ziniti [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Tuesday, October 02, 2001 11:46 Subject: [Zope-dev] file descriptors on Solaris I am running into a problem where Zope is trying to open too many file descriptors (256) in order to process a POST. I'm not sure how to phrase this question, but my reading has suggested that this limit may be set by the FILE struct in /usr/incldue/stdio.h. Does anyone know if this is used in Python/Zope? My first impression was that this problem would be easy to solve -- just up the number of FD's, but now, I'm not so sure. Any suggestions? Here's a traceback -- but I don't think it's much help Traceback (innermost last): File /u05/ilocal/opt/Zope-2.4.0-src/lib/python/ZPublisher/Publish.py, line 223, in publish_module File /u05/ilocal/opt/Zope-2.4.0-src/lib/python/ZPublisher/Publish.py, line 187, in publish File /u05/ilocal/opt/Zope-2.4.0-src/lib/python/Zope/__init__.py, line 226, in zpublisher_exception_hook (Object: ApplicationDefaultPermissions) File /u05/ilocal/opt/Zope-2.4.0-src/lib/python/ZPublisher/Publish.py, line 136, in publish File /u05/ilocal/opt/Zope-2.4.0-src/lib/python/ZPublisher/HTTPRequest.py, line 405, in processInputs File /ilocal/lib/python2.1/cgi.py, line 517, in __init__ File /ilocal/lib/python2.1/cgi.py, line 606, in read_multi File /ilocal/lib/python2.1/cgi.py, line 519, in __init__ File /ilocal/lib/python2.1/cgi.py, line 616, in read_single File /ilocal/lib/python2.1/cgi.py, line 636, in read_lines File /ilocal/lib/python2.1/cgi.py, line 723, in make_file File /ilocal/opt/Zope-2.4.0-src/lib/python/tempfile.py, line 155, in TemporaryFile OSError: [Errno 2] No such file or directory ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] file descriptors on Solaris
Yeah ... something tells me it's a little more complicated than that. Any advice on the other front? If I can reduce the number of files Zope needs to process this request, I'd grumpily agree to do that, is Zope opening a file for every input name=x.name:records? Will using input name=x_names:list help? Thanks. Andreas Jung wrote: - Original Message - From: John Ziniti [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Tuesday, October 02, 2001 12:01 Subject: Re: [Zope-dev] file descriptors on Solaris :( I was hoping against hope that this wouldn't be the answer ... I think that the hard limit onSolaris must be 256, because ulimit -n 200 seems to have the appropriate effect of making Zope complain about too many open files ... but anything higher than 256 gets a File not found ... Just ask your administrator to increase the number of file descriptors. Usually this requires *only* a reboot of the machine :-) Andreas ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] file descriptors on Solaris [SUMMARY]
I'd just like to summarize for the list some additional findings, questions and clarifications. DIAGNOSIS: It appears that this only happens when the form is specified with enctype=multipart/form-data. In that case, Zope (or, more accurately, the cgi module), tries to create temporary file for each form input, no matter what type the input has ... (I think but I can't be 100% sure about that). This seems a little weird. Why do we have to open a file for each part just because it *may* contain a file? PROGNOSIS: The problem (on Solaris) is not very easy to fix, since it lies in the system-wide definition of a file descriptor, which uses only one byte to store the fd value (i.e., anything higher than 256 is meaningless, and truncated??). Changing this struct is not easy. The problem is not that Zope is exceeding the *allowed* number of FD's (usually policed by the shell), but that Zope is exceeding the *meaningful* number of FD's. This sucks. :-) SUMMARY: If you're planning on using large forms in Zope on Solaris (version??), you'll have to move your file uploads to another page, since specifying multipart/form-data causes cgi.py to open a tempfile.TemporaryFile for each input on the form, which causes problems if the number of inputs is greater than the file descriptor address space. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] file descriptors on Solaris [SUMMARY]
This is nonsense. Solaris allows of course to use more than 256 FDs. I don't know how they are stored inside the kernel but I have been using Solaris in projects where we used 1024 FDs and more. Zope does not increase the number of allowed FDs (resource module) but inherits the settings from the environment where Zope gets started. The number of FDs are set in Solaris in /etc/system: set rlim_fd_max = 4096 set rlim_fd_cur = 1024 Solaris 7+ allows up to 65536 FDs. Solaris is definitely not my area of expertise, but to the best of my UNIX-hunting-around- looking-for-the-answer ability, that's all I can find. If Zope is started in a shell with $ ulimit -n 200 and I POST the offending form, I get a too many open files error. But if ulimit -n 512, then I get No such file or directory when $ZOPE/lib/python/tempfile.py tries to fdopen a file descriptor higher than 255 (line 155). my /etc/system doesn't have anything other than some shm settings. Can you create a form: form method=POST enctype=multipart/form-data input type=submit value=Click me to test FD's dtml-in _.range(500) input type=text name=input_dtml-var sequence-number value=dtml-var sequence-number /dtml-in /form and tell me if you get the OSError? ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )