Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Nov 28, 2006, at 6:08 PM, Guy Waugh wrote: > >> I'm still wondering whether I should be NFS-sharing the qfiles >> directory. I haven't delved into the Mailman source code to try to >> figure this out, but... > > You should be able to NFS share the qfiles directory, but you want to > be careful about how you set up your qrunners. However, this probably > won't help you with what I think you really want to do (IIUC), which > is load balance the web interface. > > First, pending messages are not kept in qfiles -- that's only for > messages that are being processed by the mail delivery subsystem. A > message that's waiting for moderation will get dequeued until it's > approved, at which point it will be re-queued into the appropriate > qfile directory. > > Access to the "databases" which manage these pending files are all > protected by Mailman's lockfile implementation, which has had a long > stable history and a high probability of being NFS-safe, modulo bugs > in specific NFS implementations of course. So as long as your web > requests can be completed within the lock timeouts, you should be able > to load-balance admindb management across multiple web servers. Of > course, while one server is accessing a list, no other processing for > that list will occur on any other machine, as those other machines > wait for the first machine's list lock to be released. However, > processing involving other lists can still occur, as can outgoing mail > delivery, which does not need to acquire a list lock. > > The story with qfiles is this: every qfile lives in its own little > slice of sha1 hash space and each hash slice is (supposed to be) owned > by exactly one qrunner process. This allows the qrunner to process > the messages in its hash slice without having to deal with pesky locks > which slows things down as contentions are serialized (a good thing > when dealing with databases, a bad thing when you're trying to churn > out a stream of messages). Thus, if you're looking to load balance > qfile directory processing, you can still do that if you assign each > qrunner process on each machine a unique slice of the hash space -- it > must be unique across all machines. IOW, machine 1 could handle the > odd slices of qfile/in while machine 2 could handle the even slices. > Or you could have 8 qrunners on each machine and slice up qfiles/in 16 > ways (the implementation requires a factor of 2 in the number of hash > slices). > > Of course, if machine 1 went down, all the messages in its hash slices > would sit unprocessed, but it would be a fairly simple matter to > reconfigure machine 2 to handle machine 1's slices, or to bring up a > fallback machine to handle those slices in the meantime. > > That's the intent anyway <wink>. I hope this makes sense and helps > you better plan your operational environment. > > - -Barry Great, thanks Barry...
So, it sounds like all information about pending posts etc. is held in the databases in the directories I'm already NFS-sharing (i.e. archives, data and lists), and the qfiles directory is specific to the qrunner running on that machine, so I don't need to worry about NFS-sharing any of the other Mailman directories. Yeehoo! Thanks again, Guy. -- Guy Waugh Unix System Administrator IT&TS, Southern Cross University Lismore, NSW, Australia Email: [EMAIL PROTECTED] Ph.: +61 2 6620 3196 Fax: +61 2 6620 3033 ------------------------------------------------------ Mailman-Users mailing list [email protected] http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org Security Policy: http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq01.027.htp
