Hi, On Wed, Jun 06, 2007 at 06:04:50PM +0200, Sascha Ottolski wrote: > I discovered a problem with the maildir implementation, that is > triggered when trying to use the QueuedDelivery with many (read: more > than current filedescriptor limit of the zope process) mails at once: ... > OSError: [Errno 24] Too many open files: > '/home/so/sandbox/var/zope/lib/python/lalisio/app/site/../../../../var/zope/var/mqueue/new'
Oh, wow. I'd forgotted that this causes the data manager to keep an open file descriptor for every email. Ouch. Suggestion: introduce IMaildirMessageWriter.close() and call it in QueuedMailDelivery.createDataManager. > I need to kill -9 the server after that, SIGTERM seems to be ignored. Ouch. > The reason for that to happen is clear to me, however, I'm not sure how > a fix could look like. Did you create an bug in launchpad? I could attach a patch that I think would fix the problem, but I don't currently have the time and energy to test myself. > One source of the problem is > > > try: > fd = os.open(filename, os.O_CREAT|os.O_EXCL|os.O_WRONLY, 0600) > except OSError: > # File exists > > > OSError is obviously to broad to catch as a test if the unique filename > could be created. May be use os.path.exists(filename) instead? But I'm > not sure if this is equivalent in a platform independant way. That solution would have a race condition. Better try: fd = os.open(filename, os.O_CREAT|os.O_EXCL|os.O_WRONLY, 0600) except OSError, e: import errno # actually put this at the very top if e.errno != errno.EEXIST: raise # File exists but would it work on Windows? Could anyone test that by running this little program on Windows: import os, errno file('tempfile', 'w').close() try: fd = os.open('tempfile', os.O_CREAT|os.O_EXCL|os.O_WRONLY, 0600) except OSError, e: print e.errno == errno.EEXIST # prints True on Linux > The real source of the problem lies in the MaildirMessageWriter: The > passed-in filedescriptor remains open until commit() is called. No > problem for sending some mails only, but when sending lots of > messages in one transaction, the filedescriptor limit kicks in. It's scary to see the problems in zope.sendmail, given that I'm at least partially responsible for all of them. > A dirty solution might be to change the write() and writeline() to open > and close the file on every call. But there's probably a saner way. Yes. > BTW, I'm wondering if it is really safe to assume that > os.rename(self._filename, self._new_filename) in the commit() message > never fails? Nothing is really safe. Renaming a file is much safer than creating a file. > Although I believe that it's almost impossible that it > really could :-) Sudden filesystem corruption, the kernel remounts it as read-only, and you cannot rename anything. But then you have bigger problems than an exception in the second phase of a commit. Marius Gedminas -- For Sale: Parachute. Only used once, never opened, small stain.
signature.asc
Description: Digital signature
_______________________________________________ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com