Re: cyrus-imapd 2.1.15, sieve, lmtpd, and return-path header
On Tue, 23 Sep 2003, Pat Lashley wrote: > Are you intentionally missing my point; or am I just not being > sufficiently clear? I'm not missing your point (I think) -- I don't agree with it. Just because lmtp is linked to sieve is no reason to assume that they are (or should be) functionally intertwined. smtp does stuff and hands off to lmtp. lmtp does stuff (figures out if the mailbox is local, adds return-path, received header) and hands off to sieve. sieve adds x-sieve header, filters, and does some action. It would be nice if filtering were completely pluggable and could be replaced by other filtering agents, much the same way that you can plug different sasl libraries. > You don't care that the value comes from the envelope because you have > become married to the idea that "it's a header visible in my mailbox and > should be filterable". You are ignoring the proper timeline. You are > ignoring the presence of a method for filtering on that data. You are > intent on having -your- solution whether it is the right one or not. No, I believe that my solution _is_ the right one. Headers added by lmtp should be visible to sieve. Things shouldn't be done to the message after sieve processes it. > > provide that header to sieve with all of the other headers it knows > > about. > > But that header isn't in the incoming message; and doesn't properly > exist until after sieve is done with it. until lmtpd is done with it. At which point sieve takes over and should have access to it. > At this point it is only a > potential header; not an actual one. Should lmtpd provide -all- > potential headers for Sieve to process? lmtpd should provide all of the headers that it creates plus all of the headers that come with a message. > And I claim that final delivery is always > > smtp -> lmtp -> sieve -> mailbox > > but that the 'sieve' step is optimized to a no-op in the case where > there is no sieve script. If that is the case, sieve should see the full set of headers as generated by lmtpd. Yet, it does not. My patch addresses that. -Chris
Re: followup: stuck lmtpd processes
On Tue, 23 Sep 2003, John Wade wrote: > Hi Andrew, > > I was the one who wrote the message you found. I finally came to the > conclusion that the flat file locking mechanism is somewhat broken in > Cyrus, but I was never a good enough C programmer to pin down what was > happening. (The mmap stuff makes it really tricky to debug.)I > wanted to blame it on the Linux kernel, but I know that others have > experienced the same problems in Solaris. > > I finally gave up and wrote a locking timeout patch for 2.0.16. see > http://www.oakton.edu/~jwade/cyrus/ for the patch and full details > > A number of other folks have tried this patch successfully on 2.0.16 and > 2.1.x, and I know it has resolved our problem. > > If you can solve the particular bug that causes this, more power to you, > if not, my work around resolves a number of possible deadlock issues. > > Enjoy, > John Hey John, Thanks for that message. If you've read a little further in your info-cyrus messages, you'll see that I apparently have hit upon a different bug than the one you found (I think). Your page was instrumental in helping me track down the source of the problem though. It turns out I had an imaps process that hung onto the lock on the user's quota file. Apparently it obtained the lock, then went off to read from the network connection and never came back. I think your patch would fix the problem where are lot of processes are contending for a lock (by making them retry), but it wouldn't help if a single process keeps the lock indefinately. Ideally it should not be possible for a process to get hung while it is holding the lock, but that will require some careful programming in this particular case. In the meantime, I'll have to keep an eye on the system. Thanks again for your debugging clues... Andy
Re: followup: stuck lmtpd processes
Hi Andrew, I was the one who wrote the message you found. I finally came to the conclusion that the flat file locking mechanism is somewhat broken in Cyrus, but I was never a good enough C programmer to pin down what was happening. (The mmap stuff makes it really tricky to debug.)I wanted to blame it on the Linux kernel, but I know that others have experienced the same problems in Solaris. I finally gave up and wrote a locking timeout patch for 2.0.16. see http://www.oakton.edu/~jwade/cyrus/ for the patch and full details A number of other folks have tried this patch successfully on 2.0.16 and 2.1.x, and I know it has resolved our problem. If you can solve the particular bug that causes this, more power to you, if not, my work around resolves a number of possible deadlock issues. Enjoy, John Andrew Morgan wrote: Following up on my previous post about stuck lmtpd processes. I found this incredibly detailed faq at: http://www.faqchest.com/prgm/cyrus-l/cyrus-01/cyrus-0111/cyrus-011102/cyrus0023_33254.html This isn't exactly the same problem, but the steps on that page helped me figure out that they are all stuck trying to get a lock on: /private/cyrus/mail/k/user/krolickp/cyrus.header Looking at /proc/locks shows: 7: POSIX ADVISORY WRITE 21903 08:11:42107658 0 EOF d23895e0 c3217f44 c510e4c4 ccbf076c 7: -> POSIX ADVISORY WRITE 32485 08:11:42107658 0 EOF ccbf0760 ee36ac44 f3bb26a4 d23895e0 ee36ac4c 7: -> POSIX ADVISORY WRITE 1802 08:11:42107658 0 EOF ee36ac40 c050ea04 ccbf0764 d23895e0 c050ea0c 7: -> POSIX ADVISORY WRITE 1217 08:11:42107658 0 EOF c050ea00 ee36a344 ee36ac44 d23895e0 ee36a34c ... I don't see how this deadlock occurred, but I'm willing to help debug it. Andy
Re: cyrus-imapd 2.1.15, sieve, lmtpd, and return-path header
On Tue, 23 Sep 2003, Pat Lashley wrote: > (And if the lmtpd authors want to contradict me here; I will claim > that messages which are not examined by Sieve should not have an > X-Sieve header added.) All messages that hit a sieve-compiled LMTPd are "processed by sieve" even if that processing is just "hey, look, there isn't a sieve script for this user... moving on..." > Hmm, I don't have the old messages; but weren't we talking about the > authors of the LMTP protocol/RFCs at this point; not the authors of > the Cyrus lmtpd? The folks who write and review RFCs tend to take > a more global view than most implementers. Heh. They're one and the same actually. RFC2033 was written by John Myers, one of the first developers on the Cyrus Project. There are a number of similar RFCs that were developed at around the same time that were definitely severely impacted by the needs of Project Cyrus. And, while RFCs are generally written to create interoperable standards, the fact is, most of the time they're written by implementors of those same standards -- the people who have the most experience getting that interoperability to work right. Indeed, often if an RFC is published and there isn't at least one full implementation of it already, glaring bugs are found after publication. (The IMAP BINARY extention has this problem, for example). > It does. The Return-Path header is not available at this point. > You are confused by the fact that a predictive copy happens to > appear in a file created by the same daemon. But it is, quite > properly, -not- available to Sieve because the message has not > yet been designated for final delivery. Forgetting about Return-path for the moment. At the minimum the X-Sieve header should be available (and probably the received header, since the message was received before it was processed by sieve). > Once again. The fact that that header is predictively inserted into > the disk file is immaterial. It is even arguably wrong. (Although > that argument is likely to fail in the face of the performance advantage > of not having to rewrite the file just to add that one header when a > final delivery is actually made.) Yeah, if we were to correct the fact that we forward messages with the Return-path, we'd correct it by skipping the first line of the file when we were writing it back to the MTA, not by writing the initial file without the Return-path header. > Procmail is basicly a hack to provide much greater capability to the > unix .forward files. Which were themselves basicly a hack. The MTA > treats it as a final delivery, whether it actually is or not. But > procmail is not well integrated into the mail system; and is most > emphaticly -not- a good model for a stand-alone sieve filter. Why not? If I had a sieve implementation that was stand-alone and I wanted it to work as a local delivery agent, I think it would make sense for it to work exactly as procmail does. And procmail is allowed to fail and exit with EX_TEMPFAIL (or other exit codes) to signal various failure conditions to the MTA. Any reasonable MTA shouldn't assume the message is delivered until that delivery agent exits with an OK status code. -Rob -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Rob Siemborski * Andrew Systems Group * Cyert Hall 207 * 412-268-7456 Research Systems Programmer * /usr/contributed Gatekeeper
Re: cyrus-imapd 2.1.15, sieve, lmtpd, and return-path header
On Tue, 23 Sep 2003, Chris Stromsoe wrote: > LMTP is not a dialect of SMTP (unless I'm reading 2033 wrong -- feel free > to correct me if I am): > >Although LMTP is an alternative protocol to ESMTP, it uses (with a >few changes) the syntax and semantics of ESMTP. This design permits >LMTP to utilize the extensions defined for ESMTP. LMTP should be >used only by specific prior arrangement and configuration, and it >MUST NOT be used on TCP port 25. I suppose whether LMTP is a "dialect of SMTP" or not is open to interpretation. It is, however, certainly part of the delivery process. And, for the record, Sieve processing happens before the 200 OK result is returned to the transmitting MTA, so technically delivery has not been completed at the time Sieve is executing. > > But presumably, conceptually at least, -after- sieve has had a whack at > > the message. Note that the absence of a sieve script for a given > > local-part does not mean that sieve doesn't process the message; only > > that it uses the default action. > > No. It is added regardless of whether or not sieve ever looks at the > message. If you have no sieve script, sieve is never invoked. Yet > x-sieve is still added. Well, if there's no sieve script, then sieve did process the message -- and did nothing to it. Note that X-Sieve is *not* added if Cyrus is compiled without sieve support. > LMTP is not a specialized dialect of SMTP. Again, open to interpretation. While LMTP is definitely *NOT* SMTP, LMTP could be described as SMTP without the requirement for the server to queue messages. > It is its own separate transport protocol that just happens to share > some of the language of smtp. smtp passing off a message to an lmtp > server is semantically the same as smtp passing it off via ftp or nntp > or whatever else. From the perspecitve of the mta, it is the functional > endpoint. Only if a 200 OK response is received. Otherwise it can have a temporary or permanent failure (just like many other delivery mechanisms). > > That's where we disagree. I claim that final delivery doesn't occur > > until it is actually placed into a mailbox. Apparently the authors of > > the lmtpd code tended more towards my view than yours. > > Hrm. Very well. I claim that when sieve is being used, final delivery is > done by sieve. When it is not in use, final delivery is done by lmtpd. > Ie, one of > > smtp -> lmtp -> mailbox > smtp -> lmtp -> sieve -> mailbox > > and not > > smtp -> lmtp -> sieve -> lmtp -> mailbox Actually, this is probably the closest of the three to reality, since lmtp has yet to issue an OK back to the mail server at the point that the script is run. Of course, a sieve failure in our implementation doesn't result in a LMTP error, it results in an attempt to deliver to the INBOX directly. And only if that fails is an LMTP error generated. Perhaps this is somewhat more accurate smtp -> lmtp -> sieve -> (failure?)-yes-> lmtp -> mailbox \ no-> (sieve actions) > It may, I don't know (I've never looked). In any event, procmail doesn't > have to take the return-path -- my mailer that calls procmail inserts a > return-path and feeds the message to procmail, which filters and delivers > my mail. It is (as far as I'm aware without doing a lot more reading than > I have time for right now) 100% normal operating behavior for sendmail > using procmail as a delivery agent. Here, procmail is the final delivery agent. That isn't really the case for Cyrus's sieve implementation. This all being said, I really am having trouble finding the Return-path header as a compelling argument to changing the behavior of the processing. The contents of the Return-path header are really a part of the envelope, and an envelope test exists (and it probably shouldn't be included in bounces/redirects). If anything, I'd think checking the version of sieve via the X-Sieve header is more interesting, but I guess that is because I fail to see the absolute need to check Return-path directly. However, I am currently leaning towards adopting (a corrected version of) the patch, since the behavior of including the headers is probably less confusing to users in general. -Rob -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Rob Siemborski * Andrew Systems Group * Cyert Hall 207 * 412-268-7456 Research Systems Programmer * /usr/contributed Gatekeeper
Re: cyrus-imapd 2.1.15, sieve, lmtpd, and return-path header
On Tue, 23 Sep 2003, Chris Stromsoe wrote:
> On Tue, 23 Sep 2003, Rob Siemborski wrote:
>
> > Ah, I missed that (or forgot). X-Sieve is added as part of
> > func->addheaders, so it might be a bit more interesting...
>
> Is addheaders used for anything other than the X-Sieve header? Was it
> ever intended to be used for more?
No.
> If it was only ever intended to be used to add the X-Sieve header, is
> there any objection to removing func->addheaders and redoing the X-Sieve
> header generation with the other headers in lmtpengine.c?
Yes. You need to remember that lmtpengine.c is used for both lmtpd and
lmtpproxyd. The former needs to add the X-Sieve header, the later does
not. This can't be done inside of lmtpengine.c.
> If it was intended to be a generic "headers to add to every message"
> interface, do you have any objection to broadening it into an array of
> struct Header {}, with code to add arbitrary headers from imapd.conf?
This wouldn't be a problem.
-Rob
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski * Andrew Systems Group * Cyert Hall 207 * 412-268-7456
Research Systems Programmer * /usr/contributed Gatekeeper
Re: cyrus-imapd 2.1.15, sieve, lmtpd, and return-path header
--On Tuesday, September 23, 2003 17:55:42 -0700 Chris Stromsoe <[EMAIL PROTECTED]> wrote: My measurements were not scientific. I'm sure that I overestimated the count of messages by some factor. My point was that envelope-to does not seem to be in wide use. All you've managed to prove is that you don't use it. As I said before, sites that -do- use it add it at the last minute before final delivery (like Return-Path). And most sites that use it immediately remove any existing Envelope-To: header on incoming messages. Any smtp server that removes a header from a message that it is relaying (I'm assuming that you mean relaying when you say "passing on") is (I'm fairly certain) violating rfc2821. Only the end points should be removing headers. I've never worked on a system that added the envelope-to header. Actually, I suspect you'll find that removing Return-Path on incomming messages is at least recommended; if not required. Since Envelope-To is non-standard; there is no standard for handling it. As I have said several times now, sites that do use it, use it in a manner analagous to the Return-Path; which dictates removing any existing version on incomming messages and only generating one for final delivery. LMTP is not a dialect of SMTP (unless I'm reading 2033 wrong -- feel free to correct me if I am): Although LMTP is an alternative protocol to ESMTP, it uses (with a few changes) the syntax and semantics of ESMTP. This design permits LMTP to utilize the extensions defined for ESMTP. LMTP should be used only by specific prior arrangement and configuration, and it MUST NOT be used on TCP port 25. I would certainly interpret "uses ... the syntax and semantics of ESMTP" as indicating that it is, in effect, a dialect of SMTP. What part of that paragraph leads you to disagree? My actual assertion is that users, given the ability to filter on the envelope sender, will also want to filter on the envelope recipient. If you use Return-Path as the handle with which to check the former; you must consider Envelope-To as the appropriate handle for checking the latter. Then write up an rfc modifying whatever the current SMTP definition is stating that mta's MUST handle the envelope-to header. Are you intentionally missing my point; or am I just not being sufficiently clear? Let me try it again. You want to provide the ability to filter on the envelope sender. Given that ability, users will want to filter on on the envelope recipient. They will expect to be able to do the later in a manner analagous to the way they do the former. That implies the use of a header and normal header conditions. Envelope-To is the de-facto standard for storing the envelope recipient in the message. There is no need for a modification or extension to SMTP because NEITHER Envelope-To nor Return-Path headers should be used for sieve filtering. That's what the envelope extension is for. Oh, and as to how widespread the use of Envelope-To is, there is at least one widely used MTA (exim) which provides a simple flag to turn it on; and one to remove existing Envelope-To: headers on incoming messages. I suspect a sizable percentage of Exim users are using it. (But you don't see that because they are using it correctly; so the header never escapes their site.) It would not surprise me to discover that other popular MTAs have easy ways to add that header. (Other than a generic mechanism for adding any header at all.) Until then, it's not a big deal in my books. You should also note that I never said that I wanted to be able to filter on the envelope sender or recipient. I want to be able to filter on the complete set of headers as they will appear in my mailbox. Regardless of what the ultimate data source is. So if your MUA adds a header, you think that Sieve should somehow be prescient enough to anticipate that and allow you to filter on it? I don't care that return-path provides the envelope values. I care that it is a header that is a part of the message that is on disk, but that is not a part of the header cache. Please forget about the details of lmtpd's implementation. That the header has been placed in the disk copy in anticipation of the likelyhood of final delivery does not change the fact that conceptually, and following the proper strictures of the RFCs, it does not yet exist when Sieve examines the message. You don't care that the value comes from the envelope because you have become married to the idea that "it's a header visible in my mailbox and should be filterable". You are ignoring the proper timeline. You are ignoring the presence of a method for filtering on that data. You are intent on having -your- solution whether it is the right one or not. But presumably, conceptually at least, -after- sieve has had a whack at the message. Note that the absence of a sieve script for a given local-part does not mean that sieve doesn't
Re: cyrus-imapd 2.1.15, sieve, lmtpd, and return-path header
On Tue, 23 Sep 2003, Pat Lashley wrote: > It wasn't clear whether you were testing incoming messages or ones > already in mailboxes. And your statement above certainly implies that a > bit less than 10% of the messages do NOT have a Return-Path header. Is > that in fact the case; or was that 40,000 an overestimate? grep ^From\ mail/* | wc -l grep ^Return-Path:\ mail/* | wc -l grep -i envelope-to mail/* | wc -l My measurements were not scientific. I'm sure that I overestimated the count of messages by some factor. My point was that envelope-to does not seem to be in wide use. > Systems that do generate Envelope-To headers generally only do so when > they believe that the message is about to achieve final delivery; and > generally remove Envelope-To headers that already existed on incoming > messages, whether or not they will be passed on or delivered locally. > (On the theory that a pre-existing header may not be trustworthy.) Any smtp server that removes a header from a message that it is relaying (I'm assuming that you mean relaying when you say "passing on") is (I'm fairly certain) violating rfc2821. Only the end points should be removing headers. I've never worked on a system that added the envelope-to header. > LMTP is a dialect of SMTP; so it hasn't left the SMTP environment until > sieve is through with it and lmtpd puts it into a cyrus mailbox. (If, in > fact, that is what sieve says is to happen to the message.) LMTP is not a dialect of SMTP (unless I'm reading 2033 wrong -- feel free to correct me if I am): Although LMTP is an alternative protocol to ESMTP, it uses (with a few changes) the syntax and semantics of ESMTP. This design permits LMTP to utilize the extensions defined for ESMTP. LMTP should be used only by specific prior arrangement and configuration, and it MUST NOT be used on TCP port 25. > > The RFC _requires_ Return-Path. It is standard. Envelope-To is > > something that somebody made up. I can't recall it being mentioned in > > any smtp rfc that I've read recently. I can't recall seeing it > > generated by any MTA. You keep asserting that it has something to do > > with this and that I should somehow "know" how it fits into the > > discussion. I don't believe that Envelope-To has anything to do with > > discussing whether or not _lmtpd should make sure that its in-memory > > data matches what it writes to disk_. > > My actual assertion is that users, given the ability to filter on the > envelope sender, will also want to filter on the envelope recipient. If > you use Return-Path as the handle with which to check the former; you > must consider Envelope-To as the appropriate handle for checking the > latter. Then write up an rfc modifying whatever the current SMTP definition is stating that mta's MUST handle the envelope-to header. Until then, it's not a big deal in my books. You should also note that I never said that I wanted to be able to filter on the envelope sender or recipient. I want to be able to filter on the complete set of headers as they will appear in my mailbox. Regardless of what the ultimate data source is. > I also assert that it is wrong to use those headers as the mechanism for > obtaining envelope values; and that the envelope extension to sieve > provides the right mechanism. I don't care that return-path provides the envelope values. I care that it is a header that is a part of the message that is on disk, but that is not a part of the header cache. > > The x-sieve header is applied regardless of sieve script processing. > > If you enable sieve with cyrus, it is _always_ added. > > But presumably, conceptually at least, -after- sieve has had a whack at > the message. Note that the absence of a sieve script for a given > local-part does not mean that sieve doesn't process the message; only > that it uses the default action. No. It is added regardless of whether or not sieve ever looks at the message. If you have no sieve script, sieve is never invoked. Yet x-sieve is still added. > > - sendmail provides a Return-Path header to procmail, the final > > delivery agent. sendmail could also provide one to lmtpd, but lmtpd > > provides its own. lmtpd is the "local mail transport protocol". > > Like a local mailer in sendmail. It is the final delivery step. > > As such, it is a specialized dialect of SMTP; and is the proper agent > for adding the Return-Path header to messages which it actually delivers > into a local mailbox. But -NOT- to messages which are bounced or > forwarded. lmtp is not a specialized dialect of smtp. It is its own separate transport protocol that just happens to share some of the language of smtp. smtp passing off a message to an lmtp server is semantically the same as smtp passing it off via ftp or nntp or whatever else. From the perspecitve of the mta, it is the functional endpoint. > I wish they would. But I strongly suspect that they had a much more > global overview of
Re: cyrus-imapd 2.1.15, sieve, lmtpd, and return-path header
On Tue, 23 Sep 2003, Rob Siemborski wrote:
> Ah, I missed that (or forgot). X-Sieve is added as part of
> func->addheaders, so it might be a bit more interesting...
Is addheaders used for anything other than the X-Sieve header? Was it
ever intended to be used for more?
If it was only ever intended to be used to add the X-Sieve header, is
there any objection to removing func->addheaders and redoing the X-Sieve
header generation with the other headers in lmtpengine.c?
If it was intended to be a generic "headers to add to every message"
interface, do you have any objection to broadening it into an array of
struct Header {}, with code to add arbitrary headers from imapd.conf?
-Chris
> -Rob
>
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> Rob Siemborski * Andrew Systems Group * Cyert Hall 207 * 412-268-7456
> Research Systems Programmer * /usr/contributed Gatekeeper
>
Re: Cyrus to USENET gateway
Ken Murchison wrote: Nils Vogels wrote: Ken Murchison wrote: Nils Vogels wrote: Hi all, I'm currently looking for some kind of solution, that will allow me to gateway between Cyrus and USENET. Right now I'm using an MLM solution, that fetches certain groups from a server, drops and drops them in a Cyrus bb, but I'm trying to see if threre is anything around that will allow me to integrate this further. The things that currently trouble me are: 1) I've found the feedcyrus shellscript, but this only seems to work for an INN install. I do not have the resources to install INN as well on the mailserver, and I do not want articles to be both in the INN system, and in the Cyrus system. Would it be possible to use for instance an slrn-pull setup for this as well ? Cyrus 2.2 has a fully functional NNTP server so you can feed articles directly into Cyrus (or pull them from another server) and have them served via IMAP/POP3 and/or NNTP. I've been playing with this option, unfortunately all the newsservers I have access to will not allow me to use the NEWNEWS command, and thus fetchnews fails miserably. Given the fact that NEWNEWS is rather expensive to run on a full-feed newsserver in busy groups, I can imagine more people having issues with this restriction. Would it be possible to have another way get the 'last fed message' other than the use of NEWNEWS ? (maybe XOVER, or something?) This would require maintaining a lot of extra state, andI really don't want to write a full featured client. Patches are always welcome. ;) Feel free to use some other tool if you find one that works. Anything that speaks NNTP or IMAP should work. OK, Ill start a search and tell you the results if I find anything useful ;-) Why not just have the newsserver feed directly to Cyrus? I only wrote fetchnews because we have our newsfeed sent to a newsserver outside our firewall and I didn't want to allow outside access to my Cyrus server. You are assuming I have control over the newsserver I use ... I do not ;-) Gr, Nils. -- Simple guidelines to happiness: Work like you don't need the money, love like your heart has never been broken and dance like no one can see you. I am testing
Re: stuck lmtpd processes
On Tue, 23 Sep 2003, Rob Siemborski wrote: > On Tue, 23 Sep 2003, Andrew Morgan wrote: > > > And that write lock was held by an imaps process. Once I killed the imaps > > process, all the lmtpd's got unstuck. Unfortunately, I realize now that > > it would have been nice to get a backtrace on that imaps process to see > > why it hadn't released the lock... :( > > Actually this is a known issue when a client hangs up in the middle of an > append. Its sort of unclear why the session doesn't always time out > properly, but: > > https://bugzilla.andrew.cmu.edu/show_bug.cgi?id=1270 > > -Rob Hmmm, did you just add comment #1 to it? :) It is good to know that I can get myself out of this easily enough, but I'd love to see this fixed in Cyrus v2.1.16 (hint, hint). :) Andy
Re: stuck lmtpd processes
On Tue, 23 Sep 2003, Andrew Morgan wrote: > Hmmm, did you just add comment #1 to it? :) Yeah. It should have been added much earlier. > It is good to know that I can get myself out of this easily enough, but > I'd love to see this fixed in Cyrus v2.1.16 (hint, hint). :) Me too, but, as the comments note, its not a trivial non-invasive fix. -Rob -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Rob Siemborski * Andrew Systems Group * Cyert Hall 207 * 412-268-7456 Research Systems Programmer * /usr/contributed Gatekeeper
Re: stuck lmtpd processes
On Tue, 23 Sep 2003, Rob Siemborski wrote: > On Tue, 23 Sep 2003, Andrew Morgan wrote: > > > I'd prefer not to restart all of cyrus because I have several hundred > > users connected right now, and this is the day that all the students are > > returning to campus. Is there a way I can kill the original lmtpd process > > that got stuck to free things up? > > Killing the lmtpd that is holding the lock should free up the rest of > them. > > Finding out where it is using gdb (if it is blocking on something else, as > opposed to having fogotten to unlock something) would be useful. Okay, a little more digging shows that it was stuck trying to get a write lock on the quota file: (gdb) bt #0 0x402ae5fb in fcntl () from /lib/libc.so.6 #1 0x08077504 in lock_reopen (fd=16, filename=0xbfffa398 "/var/spool/cyrus/config/quota/k/user.krolickp", sbuf=0xbfffa340, failaction=0xbfffa33c) at lock_fcntl.c:87 #2 0x080570b6 in mailbox_lock_quota (quota=0xbfffc6c4) at mailbox.c:1016 #3 0x08053f73 in append_setup (as=0xbfffc418, name=0xbfffb414 "user.krolickp", format=0, userid=0x0, auth_state=0x0, aclcheck=0, quotacheck=0) at append.c:209 #4 0x0804de54 in deliver_mailbox (msg=0x812ec00, stage=0x8126ce0, size=8832, flag=0x0, nflags=0, authuser=0x0, authstate=0x0, id=0x812ebd0 "<[EMAIL PROTECTED]>", user=0x812f058 "krolickp", notifyheader=0x812ec50 "From: Netflix Receiving <[EMAIL PROTECTED]>\nSubject: We've received: Michael Moore's The Awful Truth: Season 2: Disc 1 -- Rate it Now!\nTo: [EMAIL PROTECTED]", mailboxname=0xbfffc988 "INBOX", quotaoverride=0, acloverride=1) at lmtpd.c:1116 #5 0x0804e449 in deliver (msgdata=0x811bf78, authuser=0x0, authstate=0x0) at lmtpd.c:1262 #6 0x08050c11 in lmtpmode (func=0x8102040, pin=0x810f8c0, pout=0x810f910, fd=0) at lmtpengine.c:1522 #7 0x0804c707 in service_main (argc=2, argv=0x810ae48, envp=0xb990) at lmtpd.c:278 #8 0x0804c448 in main (argc=2, argv=0xb984, envp=0xb990) at service.c:508 And that write lock was held by an imaps process. Once I killed the imaps process, all the lmtpd's got unstuck. Unfortunately, I realize now that it would have been nice to get a backtrace on that imaps process to see why it hadn't released the lock... :( If I see this happen again, is there anything else I should do besides get a backtrace on the process holding the lock? Thanks for a terrific product. Aside from a few little gotchas like this, Cyrus has been a wonderful change from uw-imap. Andy
Re: stuck lmtpd processes
On Tue, 23 Sep 2003, Andrew Morgan wrote: > And that write lock was held by an imaps process. Once I killed the imaps > process, all the lmtpd's got unstuck. Unfortunately, I realize now that > it would have been nice to get a backtrace on that imaps process to see > why it hadn't released the lock... :( Actually this is a known issue when a client hangs up in the middle of an append. Its sort of unclear why the session doesn't always time out properly, but: https://bugzilla.andrew.cmu.edu/show_bug.cgi?id=1270 -Rob -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Rob Siemborski * Andrew Systems Group * Cyert Hall 207 * 412-268-7456 Research Systems Programmer * /usr/contributed Gatekeeper
Re: Cyrus to USENET gateway
Nils Vogels wrote: Ken Murchison wrote: Nils Vogels wrote: Hi all, I'm currently looking for some kind of solution, that will allow me to gateway between Cyrus and USENET. Right now I'm using an MLM solution, that fetches certain groups from a server, drops and drops them in a Cyrus bb, but I'm trying to see if threre is anything around that will allow me to integrate this further. The things that currently trouble me are: 1) I've found the feedcyrus shellscript, but this only seems to work for an INN install. I do not have the resources to install INN as well on the mailserver, and I do not want articles to be both in the INN system, and in the Cyrus system. Would it be possible to use for instance an slrn-pull setup for this as well ? Cyrus 2.2 has a fully functional NNTP server so you can feed articles directly into Cyrus (or pull them from another server) and have them served via IMAP/POP3 and/or NNTP. I've been playing with this option, unfortunately all the newsservers I have access to will not allow me to use the NEWNEWS command, and thus fetchnews fails miserably. Given the fact that NEWNEWS is rather expensive to run on a full-feed newsserver in busy groups, I can imagine more people having issues with this restriction. Would it be possible to have another way get the 'last fed message' other than the use of NEWNEWS ? (maybe XOVER, or something?) This would require maintaining a lot of extra state, andI really don't want to write a full featured client. Patches are always welcome. ;) Feel free to use some other tool if you find one that works. Anything that speaks NNTP or IMAP should work. Why not just have the newsserver feed directly to Cyrus? I only wrote fetchnews because we have our newsfeed sent to a newsserver outside our firewall and I didn't want to allow outside access to my Cyrus server. -- Kenneth Murchison Oceana Matrix Ltd. Software Engineer 21 Princeton Place 716-662-8973 x26 Orchard Park, NY 14127 --PGP Public Key--http://www.oceana.com/~ken/ksm.pgp
Re: EX_TEMPFAIL
On Tue, 23 Sep 2003, Andrew Brink wrote: > I am getting many: cyrus mailer (/usr/cyrus/bin/deliver) exited with > EX_TEMPFAIL > > Can someone provide some inside on what this means? EX_TEMPFAIL means temporary failure delivering mail. In practice, that usually means over quota. Andy
Re: cyrus-imapd 2.1.15, sieve, lmtpd, and return-path header
--On Tuesday, September 23, 2003 13:36:18 -0700 Chris Stromsoe <[EMAIL PROTECTED]> wrote: On Tue, 23 Sep 2003, Pat Lashley wrote: --On Tuesday, September 23, 2003 02:18:50 -0700 Chris Stromsoe <[EMAIL PROTECTED]> wrote: > out of ~40,000 messages I've got handy, between 50 and 60 had some > variant of envelope-to (either plain or with x- or old-), and 36,906 > had a return-path header. Return-Path is an rfc mandated header that > must be handled. None of the messages should have had either of those headers when you recieved them. It's very unlikely that any of the messages did have a Return-Path when they were handed to me. I wasn't saying that they did. It wasn't clear whether you were testing incoming messages or ones already in mailboxes. And your statement above certainly implies that a bit less than 10% of the messages do NOT have a Return-Path header. Is that in fact the case; or was that 40,000 an overestimate? All of the messages containing envelope-to variants came that way. (And, in fact, there is no reason why they shouldn't -- as you said yourself, envelop-to is not a standard header; it could be inserted at either end.) Systems that do generate Envelope-To headers generally only do so when they believe that the message is about to achieve final delivery; and generally remove Envelope-To headers that already existed on incoming messages, whether or not they will be passed on or delivered locally. (On the theory that a pre-existing header may not be trustworthy.) Yes, the RFC mandates a Return-Path header; but it specificly says that it is to be generated only at the point of final delivery into a file. Keep reading the RFC. It also says that the message is considered delivered when it leaves the SMTP environment. Ie, when lmtpd takes it, lmtpd should add the Return-Path header. LMTP is a dialect of SMTP; so it hasn't left the SMTP environment until sieve is through with it and lmtpd puts it into a cyrus mailbox. (If, in fact, that is what sieve says is to happen to the message.) This seems very much like you're sniping at me. I almost expected the sentence to end with ", junior" or something similar. It's nice that you have a lot of experience doing software development and that you bring your experience to play. However, I don't believe that your experience has anything to do with this issue. I'm sorry that you feel that way. I was merely trying to point out that you are focusing on one half of the problem; and that your proposed 'solution' fails to address the other half. Or even admit to its existance. As to the applicability of my experience; that is what helps me to step back and look at the big picture to identify the actual requirements rather than the immediate request. The RFC _requires_ Return-Path. It is standard. Envelope-To is something that somebody made up. I can't recall it being mentioned in any smtp rfc that I've read recently. I can't recall seeing it generated by any MTA. You keep asserting that it has something to do with this and that I should somehow "know" how it fits into the discussion. I don't believe that Envelope-To has anything to do with discussing whether or not _lmtpd should make sure that its in-memory data matches what it writes to disk_. My actual assertion is that users, given the ability to filter on the envelope sender, will also want to filter on the envelope recipient. If you use Return-Path as the handle with which to check the former; you must consider Envelope-To as the appropriate handle for checking the latter. I also assert that it is wrong to use those headers as the mechanism for obtaining envelope values; and that the envelope extension to sieve provides the right mechanism. The x-sieve header is applied regardless of sieve script processing. If you enable sieve with cyrus, it is _always_ added. But presumably, conceptually at least, -after- sieve has had a whack at the message. Note that the absence of a sieve script for a given local-part does not mean that sieve doesn't process the message; only that it uses the default action. - lmtpd has written the received header to disk. When that message is forwarded or bounced the on disk file is sent. If on-disk format is canonical, then the header cache needs to have the headers too. If the header cache is canonical, then the on-disk representation needs to not have those headers until the message is dispensed with. In the case of the Recieved: header, the bounce or forward happens after lmtp has 'accepted' the message for processing; so it is correct to include that header in the bounced or forwarded message even if it is not correct to have it yet when sieve processes it. (See my previous statement that argues that the Recieved: header should only be added once lmtpd indicates to the MTA that it has accepted the message for processing.) A similar argument applies to the X-Sieve: header - by the time the message
Re: cyrus-imapd 2.1.15, sieve, lmtpd, and return-path header
On Tue, 23 Sep 2003, Chris Stromsoe wrote: > > You are assuming that sieve processing occurs -AFTER- final delivery. > > The authors of lmtpd apparently consider it to be happening -BEFORE- > > final delivery; with the sieve script itself determining whether there > > will even BE a final delivery at that point. > > Or, the authors of lmtpd never considered this situation and it wasn't a > big deal having or not having the missing headers in the cache. I'll let > them speak for themselves. This is very likely much closer to the truth. -Rob -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Rob Siemborski * Andrew Systems Group * Cyert Hall 207 * 412-268-7456 Research Systems Programmer * /usr/contributed Gatekeeper
Re: EX_TEMPFAIL
Andrew Brink wrote: I am getting many: cyrus mailer (/usr/cyrus/bin/deliver) exited with EX_TEMPFAIL Can someone provide some inside on what this means? EX_TEMPFAIL means a temporary error, which is expected to dissolve within time. Nice examples are locking errors (trying to lock an already locked mailbox) or quota-errors (trying to deliver to a mailbox which is full) HTH & HAND!
Re: Multiple IMAP Connections
On Tue, 23 Sep 2003, Andrew Brink wrote: > Do you guys see any problems that can occur from multiple users at > different locations check the same IMAP mailbox? (With getting the > folder/message list in particular and maintaining their connections.) > > We have a customer here that has a ~500MB IMAP Box, and they have > different people use this box in different locations. Any Issues with > this? That's a pretty large mailbox, but nothing in the design of cyrus prevents multiple users from using it. (there might be some delays due to the amount of IO the index/cache files will require, but...) -Rob -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Rob Siemborski * Andrew Systems Group * Cyert Hall 207 * 412-268-7456 Research Systems Programmer * /usr/contributed Gatekeeper
Re: stuck lmtpd processes
On Tue, 23 Sep 2003, Andrew Morgan wrote: > I'd prefer not to restart all of cyrus because I have several hundred > users connected right now, and this is the day that all the students are > returning to campus. Is there a way I can kill the original lmtpd process > that got stuck to free things up? Killing the lmtpd that is holding the lock should free up the rest of them. Finding out where it is using gdb (if it is blocking on something else, as opposed to having fogotten to unlock something) would be useful. -Rob -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Rob Siemborski * Andrew Systems Group * Cyert Hall 207 * 412-268-7456 Research Systems Programmer * /usr/contributed Gatekeeper
EX_TEMPFAIL
I am getting many: cyrus mailer (/usr/cyrus/bin/deliver) exited with EX_TEMPFAIL Can someone provide some inside on what this means? Andrew Brink, CCNA, WCSP NetStandard, Inc. 913-262-3888
Re: cyrus-imapd 2.1.15, sieve, lmtpd, and return-path header
On Tue, 23 Sep 2003, Chris Stromsoe wrote: > I sent a more complete patch to devel-cyrus to cache both the Received and > Return-Path headers. It is not writing out the X-Sieve header, though I > can modify to cache that as well. As far as I am aware, those are the > only three that are generated. Ah, I missed that (or forgot). X-Sieve is added as part of func->addheaders, so it might be a bit more interesting... -Rob -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Rob Siemborski * Andrew Systems Group * Cyert Hall 207 * 412-268-7456 Research Systems Programmer * /usr/contributed Gatekeeper
Re: cyrus-imapd 2.1.15, sieve, lmtpd, and return-path header
On Tue, 23 Sep 2003, Pat Lashley wrote: > You are assuming that sieve processing occurs -AFTER- final delivery. > The authors of lmtpd apparently consider it to be happening -BEFORE- > final delivery; with the sieve script itself determining whether there > will even BE a final delivery at that point. > > And you should also remember that there's nothing about sieve that > restricts it to delivery-time or post-delivery-time. It could just > as easily be integrated into an MTA for relay-time processing; or > into an MUA for use during the send. I tend to agree -- at the point sieve is processing the message, final delivery has not occured, and thus the headers that are being refered to here don't really exist yet. In some ways it is a judgement call. In any case, the suggested patch would need to handle all of the potential headers, not just Return-path. (I still haven't quite settled on wether to take it or not though). -Rob -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Rob Siemborski * Andrew Systems Group * Cyert Hall 207 * 412-268-7456 Research Systems Programmer * /usr/contributed Gatekeeper
Re: cyrus-imapd 2.1.15, sieve, lmtpd, and return-path header
On Tue, 23 Sep 2003, Rob Siemborski wrote: > In any case, the suggested patch would need to handle all of the > potential headers, not just Return-path. (I still haven't quite settled > on wether to take it or not though). I sent a more complete patch to devel-cyrus to cache both the Received and Return-Path headers. It is not writing out the X-Sieve header, though I can modify to cache that as well. As far as I am aware, those are the only three that are generated. -Chris
Re: Cyrus to USENET gateway
Ken Murchison wrote: Nils Vogels wrote: Hi all, I'm currently looking for some kind of solution, that will allow me to gateway between Cyrus and USENET. Right now I'm using an MLM solution, that fetches certain groups from a server, drops and drops them in a Cyrus bb, but I'm trying to see if threre is anything around that will allow me to integrate this further. The things that currently trouble me are: 1) I've found the feedcyrus shellscript, but this only seems to work for an INN install. I do not have the resources to install INN as well on the mailserver, and I do not want articles to be both in the INN system, and in the Cyrus system. Would it be possible to use for instance an slrn-pull setup for this as well ? Cyrus 2.2 has a fully functional NNTP server so you can feed articles directly into Cyrus (or pull them from another server) and have them served via IMAP/POP3 and/or NNTP. I've been playing with this option, unfortunately all the newsservers I have access to will not allow me to use the NEWNEWS command, and thus fetchnews fails miserably. Given the fact that NEWNEWS is rather expensive to run on a full-feed newsserver in busy groups, I can imagine more people having issues with this restriction. Would it be possible to have another way get the 'last fed message' other than the use of NEWNEWS ? (maybe XOVER, or something?) 2) When people put new postings in the BB, I would like some way to gate it out into the USENET world again. For this setup, I have no clues ... :/ As Rob said, lmtp2nntp is what you want: http://www.ossp.org/pkg/tool/lmtp2nntp/ Works like a charm, thanks ;-) Greetings, Nils.
Re: ipurge as an event in cyrus.conf
On Tue, 23 Sep 2003, Lars Peterson wrote: > # purge trash every day at midnight > purgetrashcmd="ipurge -f -d 1 user/*/Trash" at= period=1440 > > # purge spam every 3 days at 1AM > purgespam cmd="ipurge -f -d 3 user/*/Spam" at=0100 period=43212 Drop that period= mbxclean001 cmd="/usr/sbin/ipurge -s -d 15 ima.root ima.postmaster.s pam ima.postmaster.spamquarantine ima.postmaster.virus" at=0415 Works here. -- "One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie." -- The Silicon Valley Tarot Henrique Holschuh
Re: cyrus-imapd 2.1.15, sieve, lmtpd, and return-path header
On Tue, 23 Sep 2003, Pat Lashley wrote: > --On Tuesday, September 23, 2003 02:18:50 -0700 Chris Stromsoe > <[EMAIL PROTECTED]> wrote: > > > out of ~40,000 messages I've got handy, between 50 and 60 had some > > variant of envelope-to (either plain or with x- or old-), and 36,906 > > had a return-path header. Return-Path is an rfc mandated header that > > must be handled. > > None of the messages should have had either of those headers when you > recieved them. It's very unlikely that any of the messages did have a Return-Path when they were handed to me. I wasn't saying that they did. All of the messages containing envelope-to variants came that way. (And, in fact, there is no reason why they shouldn't -- as you said yourself, envelop-to is not a standard header; it could be inserted at either end.) > Yes, the RFC mandates a Return-Path header; but it specificly says > that it is to be generated only at the point of final delivery into > a file. Keep reading the RFC. It also says that the message is considered delivered when it leaves the SMTP environment. Ie, when lmtpd takes it, lmtpd should add the Return-Path header. > >> I keep mentioning Envelope-To because 30+ years of Software > >> Engineering experience tells me that if you handle Return-Path, > >> people will also want Envelope-To. > >I'm still not sure how Envelope-To applies to this discussion. > > Please re-read the first quoted paragraph above. Repeat until you > DO understand how Envelope-To applies to this discussion... This seems very much like you're sniping at me. I almost expected the sentence to end with ", junior" or something similar. It's nice that you have a lot of experience doing software development and that you bring your experience to play. However, I don't believe that your experience has anything to do with this issue. The RFC _requires_ Return-Path. It is standard. Envelope-To is something that somebody made up. I can't recall it being mentioned in any smtp rfc that I've read recently. I can't recall seeing it generated by any MTA. You keep asserting that it has something to do with this and that I should somehow "know" how it fits into the discussion. I don't believe that Envelope-To has anything to do with discussing whether or not _lmtpd should make sure that its in-memory data matches what it writes to disk_. > > On Tue, 23 Sep 2003, Pat Lashley wrote: > > > >> --On Monday, September 22, 2003 22:40:00 -0700 Chris Stromsoe > >> <[EMAIL PROTECTED]> wrote: > >> > >> > That, imho, is not correct behavior. > >> > >> What headers, aside from Return-Path, are not cached in RAM? > > > > The Received header that lmtpd adds and the X-Sieve header. There may be > > others in other areas of the code, but those two and the Return-Path > > header are the three that I saw. > > Well, it makes sense taht the X-Sieve header wouldn't be applied until > sieve has processed the message. The x-sieve header is applied regardless of sieve script processing. If you enable sieve with cyrus, it is _always_ added. > As to lmtpd's Received header; I could certainly see an argument for > including it if lmtpd has already sent back a positive response to the > MTA. However, if LMTP runs sieve while the LMTP connection is still > open, then it is right to withold that header until it has actually > accepted the message. Two things: - lmtpd has written the received header to disk. When that message is forwarded or bounced the on disk file is sent. If on-disk format is canonical, then the header cache needs to have the headers too. If the header cache is canonical, then the on-disk representation needs to not have those headers until the message is dispensed with. - sendmail provides a Return-Path header to procmail, the final delivery agent. sendmail could also provide one to lmtpd, but lmtpd provides its own. lmtpd is the "local mail transport protocol". Like a local mailer in sendmail. It is the final delivery step. Sure, other things can happen after that; those things have nothing to do with anything that happened up to that point. When sieve forwards or rejects a message, it invokes sendmail as a client. It doesn't try to relay the message through the next hop or anything like that. It reinjects the message. > > There is no RFC or draft that says that the return-path header should > > not be visible as a header to sieve scripts, whereas 2821 does say > > that the final delivery must add the return-path header. To me, that > > says that return-path should be added by lmtpd and be visible to > > sieve. > > You are assuming that sieve processing occurs -AFTER- final delivery. > The authors of lmtpd apparently consider it to be happening -BEFORE- > final delivery; with the sieve script itself determining whether there > will even BE a final delivery at that point. Or, the authors of lmtpd never considered this situation and it wasn't a big deal having or not having the missing headers in th
Multiple IMAP Connections
Title: Multiple IMAP Connections Do you guys see any problems that can occur from multiple users at different locations check the same IMAP mailbox? (With getting the folder/message list in particular and maintaining their connections.) We have a customer here that has a ~500MB IMAP Box, and they have different people use this box in different locations. Any Issues with this? Andrew Brink, CCNA, WCSP NetStandard, Inc. 913-262-3888
Re: ipurge as an event in cyrus.conf
Lars Peterson wrote: Hello all, Wondering if anyone has had any success with using ipurge events in the cyrus.conf file... I would like to periodically purge all users Spam and Trash folders and was thinking something like this would work in cyrus.conf (vs. using cron): # purge trash every day at midnight purgetrashcmd="ipurge -f -d 1 user/*/Trash" at= period=1440 # purge spam every 3 days at 1AM purgespam cmd="ipurge -f -d 3 user/*/Spam" at=0100 period=43212 The 'at' option (if available in 2.1) and 'period' option are mutually exclusive. If you use 'at', the command will run at that time every day. If you use 'period', it will run at server starup and then at every interval after that. I used to use ipurge as an event in cyrus.conf for expiring usenet articles in v2.2 until i wrote cyr_expire. -- Kenneth Murchison Oceana Matrix Ltd. Software Engineer 21 Princeton Place 716-662-8973 x26 Orchard Park, NY 14127 --PGP Public Key--http://www.oceana.com/~ken/ksm.pgp
Fatal Error from chk_cyrus
I'm getting a fatal error when trying to check a mailbox. /usr/cyrus/bin/chk_cyrus -M [EMAIL PROTECTED] Examining mailbox: [EMAIL PROTECTED] bad mailbox [EMAIL PROTECTED] in chkmbox fatal error: fatal error I don't think this problem is isolated to the one mailbox though. I also cannot add new mailboxes to this server. When using cyradm to create the mailbox it doesn't error and acts like it creates the mailbox. It also appears to create the directory: find /var/spool/imap -name /var/spool/imap/domain/R/starband.net/A/user/ In addition when I try to give permissions to cyrus to delete the [EMAIL PROTECTED] mailbox cyradm gives me a "System I/O Error" Thanks for any help. Please let me know if you need more information. -- Hank Beatty
ipurge as an event in cyrus.conf
Hello all, Wondering if anyone has had any success with using ipurge events in the cyrus.conf file... I would like to periodically purge all users Spam and Trash folders and was thinking something like this would work in cyrus.conf (vs. using cron): # purge trash every day at midnight purgetrashcmd="ipurge -f -d 1 user/*/Trash" at= period=1440 # purge spam every 3 days at 1AM purgespam cmd="ipurge -f -d 3 user/*/Spam" at=0100 period=43212 I ran the ipurge commands as user cyrus from the command line and they seemed to have worked fine. But when I stuck the above lines in my cyrus.conf file and restarted cyrus, I haven't seen any auto purging... RedHat 9 w/cyrus-imapd-2.1.13 & cyrus-sasl-2.1.13 Any ideas? Lars Peterson SysAdmin / Programmer iTECH Inc.
followup: stuck lmtpd processes
Following up on my previous post about stuck lmtpd processes. I found this incredibly detailed faq at: http://www.faqchest.com/prgm/cyrus-l/cyrus-01/cyrus-0111/cyrus-011102/cyrus0023_33254.html This isn't exactly the same problem, but the steps on that page helped me figure out that they are all stuck trying to get a lock on: /private/cyrus/mail/k/user/krolickp/cyrus.header Looking at /proc/locks shows: 7: POSIX ADVISORY WRITE 21903 08:11:42107658 0 EOF d23895e0 c3217f44 c510e4c4 ccbf076c 7: -> POSIX ADVISORY WRITE 32485 08:11:42107658 0 EOF ccbf0760 ee36ac44 f3bb26a4 d23895e0 ee36ac4c 7: -> POSIX ADVISORY WRITE 1802 08:11:42107658 0 EOF ee36ac40 c050ea04 ccbf0764 d23895e0 c050ea0c 7: -> POSIX ADVISORY WRITE 1217 08:11:42107658 0 EOF c050ea00 ee36a344 ee36ac44 d23895e0 ee36a34c ... I don't see how this deadlock occurred, but I'm willing to help debug it. Andy
Re: single quote in mailbox
On Tue, 23 Sep 2003, Etienne Goyer wrote: > On Tue, Sep 23, 2003 at 12:32:56PM -0400, Rob Siemborski wrote: > > GOODCHARS is designed by default to be very conservative about what it > > allows -- we could include a number of characters that we don't because > > they are bad in various environemnts (especially the shell -- this > > especially comes up when working with the mail store directly). > > > > I'm pretty sure there isn't any harm in including ', though I don't think > > we'll be doing so in the base distribution. > > Could GOODCHARS be user-defined, in imapd.conf for example ? It could > default to the current constant. IIRC, the RFC only forbid NUL and your > hierarchy separator, so theorically any other character could be > acceptable. I worry about doing this -- 8bit characters, for example, are still not allowed. If an implementation of such was done carefully to only allow users to specify "questionable" characters as allowed, it'd probably be ok. I don't want to commit to it without seeing the patch though. -Rob -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Rob Siemborski * Andrew Systems Group * Cyert Hall 207 * 412-268-7456 Research Systems Programmer * /usr/contributed Gatekeeper
Re: cyrus-imapd 2.1.15, sieve, lmtpd, and return-path header
--On Tuesday, September 23, 2003 02:18:50 -0700 Chris Stromsoe <[EMAIL PROTECTED]> wrote: On Tue, 23 Sep 2003, Pat Lashley wrote: --On Monday, September 22, 2003 22:40:00 -0700 Chris Stromsoe <[EMAIL PROTECTED]> wrote: I keep mentioning Envelope-To because 30+ years of Software Engineering experience tells me that if you handle Return-Path, people will also want Envelope-To. out of ~40,000 messages I've got handy, between 50 and 60 had some variant of envelope-to (either plain or with x- or old-), and 36,906 had a return-path header. Return-Path is an rfc mandated header that must be handled. None of the messages should have had either of those headers when you recieved them. Yes, the RFC mandates a Return-Path header; but it specificly says that it is to be generated only at the point of final delivery into a file. I'm still not sure how Envelope-To applies to this discussion. Please re-read the first quoted paragraph above. Repeat until you DO understand how Envelope-To applies to this discussion... > That, imho, is not correct behavior. What headers, aside from Return-Path, are not cached in RAM? The Received header that lmtpd adds and the X-Sieve header. There may be others in other areas of the code, but those two and the Return-Path header are the three that I saw. Well, it makes sense taht the X-Sieve header wouldn't be applied until sieve has processed the message. As to lmtpd's Received header; I could certainly see an argument for including it if lmtpd has already sent back a positive response to the MTA. However, if LMTP runs sieve while the LMTP connection is still open, then it is right to withold that header until it has actually accepted the message. > I don't what the Sieve RFC says about return-path. I'm pretty sure > that it isn't germane to this topic. Then why was it necessary to create an "envelope" extension to sieve? I don't know. Maybe early versions of lmtpd weren't following rfc2821 and properly settings the return-path, and sieve couldn't get at the information any other way. Maybe the envelope extension was added to get at envelope-to and envelope-from was added as an afterthought. Or, maybe, it is just more correct to think of and process the envelope as separate from the in-message headers. Nonsense. I'm only suggesting that it be smart enough to actually follow the RFCs and drafts; and to know about a few of the extensions like envelope. There is no RFC or draft that says that the return-path header should not be visible as a header to sieve scripts, whereas 2821 does say that the final delivery must add the return-path header. To me, that says that return-path should be added by lmtpd and be visible to sieve. You are assuming that sieve processing occurs -AFTER- final delivery. The authors of lmtpd apparently consider it to be happening -BEFORE- final delivery; with the sieve script itself determining whether there will even BE a final delivery at that point. And you should also remember that there's nothing about sieve that restricts it to delivery-time or post-delivery-time. It could just as easily be integrated into an MTA for relay-time processing; or into an MUA for use during the send. Making smartsieve -- and every other application out there that generates sieve scripts -- work around lmtpd's shortcomings doesn't make any sense to me. Fixing lmtpd so that the in-ram cache mirrors the on-disk data store does. It makes no sense to you because you continue to believe that it is a shortcoming and not a correct behavour. Stop trying to justify your 'solution' and try to take an objective look at the problem. You will find that it is solved admirably by the envelope extension. You're trying to change LMTP to provide a header that WILL NOT EXIST in other sieve environments. It will not be at all obvious to the naive user that that header was inserted at the last minute rather than following the message from its inception. I still don't see what it is that you are trying to say. You seem to be assuming some sort of lmtp/sieve interdependance. Please correct me if I'm wrong. There is an interdependance -in this implementation- because lmtpd is the agent that invokes the sieve processing. Your mistake is in considering the sieve processing to be done -after- lmtpd performs final delivery. In actual fact, it is done once lmtpd has accepted the message but -before- it determines whether there will even be a final delivery. It cannot know whether there will be a final at this point until the sieve script has been run. Therefore it MUST NOT (in RFC terminology) add the Return-Path header before sieve processing. As far as being not obvious to the naive user ... when has that stopped anything from happening? Wasn't the basis for your original 'problem' that naive users could not understand why they couldn't filter on the Return-Path header? You are confusing functionality with implementa
Exim and Cyrus shared folders
Hello - I am running debian
testing with Exim (3.36) and Cyrus Imap (1.5.19). I'm
having difficulty configuring exim to deliver mail to a
sharedfolder. I created a shared folder with
cyradm, like this:linux:/var/spool/cyrus/mail# cyradm -u mailadmin
127.0.0.1127.0.0.1 password:127.0.0.1> cm shared127.0.0.1> sam
shared anyone lrswipcda127.0.0.1> lam sharedanyone
lrswipcda I can use cyrdeliver to send email to the
folder:linux:/var/spool/cyrus/mail# cat sample_mail |cyrdeliver -m
shared However, this will not
work:linux:/var/spool/cyrus/mail# mail shared -s "Sample" <
sample_mail Neither does
thislinux:/var/spool/cyrus/mail# mail bb+shared -s "Sample" <
sample_mail They both result in error messages in the
/var/log/exim/mainlog: ** shared: unknown local-part "shared" in domain
"linux" Here is the transport section of
exim.conflocal_delivery: driver = pipe command =
"/usr/sbin/cyrdeliver ${local_part}" return_path_add
return_output prefix = "" user =
cyrus Am I missing something else in the
exim.conf? Please cc: me directly on
responses.
Re: Cyrus IMAP use questions.
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Denis V. Suhanov wrote: | Hello all, | | I am asking because | another problem I have here is letting users change their sasldb-based | passwords without having to log in to the box. look at MailAdmin http://mailadmin.sf.net/ for a php interface to the sasldb database. I just implemented it on a Debian testing box with sasl2 and unixhierachysep and had to make some modifications so it will be a while before I can submit another set of patches to handle this scenario. - -- James A. Pattie [EMAIL PROTECTED] Linux -- SysAdmin / Programmer Xperience, Inc. http://www.pcxperience.com/ http://www.xperienceinc.com/ GPG Key Available at http://www.pcxperience.com/gpgkeys/james.html -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.3 (GNU/Linux) Comment: Using GnuPG with Debian - http://enigmail.mozdev.org iD8DBQE/cJBhtUXjwPIRLVERAp3eAKDv9+q8PRm/rBCmWrmK3L+UCIgR+QCgzPG6 vtuKMREgr2qShgqoV4Iv8pA= =dqCa -END PGP SIGNATURE- -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. Mailscanner thanks transtec Computers for their support.
Re: Cyrus IMAP use questions.
On Tue, Sep 23, 2003 at 09:56:26AM -0700, Denis V. Suhanov wrote: > First of all, I wanted to implement incoming mail sorting via > procmail. Considering that none of my users have local shell accounts, > what is the right way to implementing individual procmail scripts for > those users who don't want to use sieve? Or should I use sieve as an > alternative? My only concern with it is that most of the clients my > users use are unaware of sieve and I have no clue how to provide them > an access to their sieve's scripts in this case. Give them a web interface to Sieve. Websieve is an option, as would the Ingo module of the Horde framework. I do not think it would possible to use procmail with Cyrus. Is it possible to do MTA -> procmail -> Cyrus deliver ? > I also was asked to provide an ability to collect mail from other > user's accounts automatically (mostly pop3) and to put it in my IMAP > server's folders. Fetchmail seems to be useful for that, but the same > problem appears - since my users do not have local accounts, I am not > sure what would be a right way to create fetchmail's configuration > files for each of them individually. I am thinking of providing > something like an SSL-wrapped web page with authentication against > sasldb for this, but still you might have a better idea on that. > Speaking of this approach, is there a standard tool for this kind of > authentication (SSL-ed webpage and sasldb)? I am asking because > another problem I have here is letting users change their sasldb-based > passwords without having to log in to the box. Instead of reinventing the wheel, you might want to look into the CVS HEAD version of the IMP webmail (also part of the Horde framework). It have a feature to fetch mail from other account over POP3, IMAP, etc. With IMP "fetchmail" feature and the Ingo Sieve script manager, you should really try Horde (www.horde.org ... get a nightly snapshot for the latest features). Not a flame, but if possible you should turn off plain-text justification. It make your email very hard to read in mono-spaced font (console mail reader). If this a feature of The Bat! ? -- Etienne GoyerLinux Québec Technologies Inc. http://www.LinuxQuebec.com [EMAIL PROTECTED]
stuck lmtpd processes
I logged onto my cyrus mail server today and found that I have about 500
lmtpd process that seem to be stuck trying to deliver mail to a single
user's inbox.
I am accepting mail directly to cyrus' lmtpd from a separate machine
running postfix, which is our campus mail relay. The following is the
message I see in the postfix logs for all delivery attempts after 9:07am
yesterday:
Sep 22 09:16:56 smtp2 postfix/lmtp[30332]: 4F275170085: to=<[EMAIL PROTECTED]>,
relay=mail1.onid.oregonstate.edu[128.193.4.128], delay=600, status=deferred
(conversation with mail1.onid.oregonstate.edu[128.193.4.128] timed out while sending
end of data -- message may be sent more than once)
All of the lmtpd processes seem to be stuck trying to obtain a lock:
[EMAIL PROTECTED] config]# strace -p 27332
fcntl64(13, F_SETLKW, {type=F_WRLCK, whence=SEEK_SET, start=0, len=0}
And here is the lsof output from one of these stuck lmtpd processes:
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
lmtpd 27332 cyrus cwdDIR8,2 40962 /
lmtpd 27332 cyrus rtdDIR8,2 40962 /
lmtpd 27332 cyrus txtREG8,2 1562302 3014706 /usr/local/cyrus/bin/lmtpd
lmtpd 27332 cyrus memREG8,290210 180307 /lib/ld-2.2.5.so
lmtpd 27332 cyrus memREG8,2 26590032792
/usr/local/lib/libsasl2.so.2.0.15
lmtpd 27332 cyrus memREG 8,17 8192 11665411
/private/cyrus/config/db/__db.001
lmtpd 27332 cyrus memREG8,2 182084 2064435 /usr/lib/libssl.so.0.9.6
lmtpd 27332 cyrus memREG8,2 771908 2064434
/usr/lib/libcrypto.so.0.9.6
lmtpd 27332 cyrus memREG8,256480 180323 /lib/libresolv-2.2.5.so
lmtpd 27332 cyrus memREG8,2 684092 2064403 /usr/lib/libdb3.so.3.0.2
lmtpd 27332 cyrus memREG8,224328 180305 /lib/libwrap.so.0.7.6
lmtpd 27332 cyrus memREG8,269472 180315 /lib/libnsl-2.2.5.so
lmtpd 27332 cyrus memREG8,2 1153784 180310 /lib/libc-2.2.5.so
lmtpd 27332 cyrus memREG8,2 8008 180313 /lib/libdl-2.2.5.so
lmtpd 27332 cyrus memREG8,298114 3031068
/usr/local/lib/sasl2/libsasldb.so.2.0.15
lmtpd 27332 cyrus memREG8,275762 3031072
/usr/local/lib/sasl2/libplain.so.2.0.15
lmtpd 27332 cyrus memREG8,219136 180311 /lib/libcrypt-2.2.5.so
lmtpd 27332 cyrus memREG8,274972 3031076
/usr/local/lib/sasl2/libanonymous.so.2.0.15
lmtpd 27332 cyrus memREG 8,17 270336 11665412
/private/cyrus/config/db/__db.002
lmtpd 27332 cyrus memREG 8,1798304 11665413
/private/cyrus/config/db/__db.003
lmtpd 27332 cyrus memREG 8,17 16064512 11665414
/private/cyrus/config/db/__db.004
lmtpd 27332 cyrus memREG 8,1724576 11665415
/private/cyrus/config/db/__db.005
lmtpd 27332 cyrus memREG 8,17 11562604 9797758
/private/cyrus/config/mailboxes.db
lmtpd 27332 cyrus memREG 8,17 167 42107658
/private/cyrus/mail/k/user/krolickp/cyrus.header
lmtpd 27332 cyrus memREG8,232668 180318 /lib/libnss_files-2.2.5.so
lmtpd 27332 cyrus memREG8,212176 180317 /lib/libnss_dns-2.2.5.so
lmtpd 27332 cyrus memREG8,240152 180316
/lib/libnss_compat-2.2.5.so
lmtpd 27332 cyrus0u IPv4 55681432 TCP
mail1.onid.oregonstate.edu:cfinger->smtp2.oregonstate.edu:39183 (CLOSE_WAIT)
lmtpd 27332 cyrus1u IPv4 55681432 TCP
mail1.onid.oregonstate.edu:cfinger->smtp2.oregonstate.edu:39183 (CLOSE_WAIT)
lmtpd 27332 cyrus2u IPv4 55681432 TCP
mail1.onid.oregonstate.edu:cfinger->smtp2.oregonstate.edu:39183 (CLOSE_WAIT)
lmtpd 27332 cyrus3w FIFO0,5 978 pipe
lmtpd 27332 cyrus4u IPv4977 TCP *:cfinger (LISTEN)
lmtpd 27332 cyrus5u REG 8,17 10484911 11665942
/private/cyrus/config/db/log.22 (deleted)
lmtpd 27332 cyrus6r REG 8,17 10484911 11665942
/private/cyrus/config/db/log.22 (deleted)
lmtpd 27332 cyrus7u REG 8,17 7716864 9797635
/private/cyrus/config/deliver.db
lmtpd 27332 cyrus8u REG 8,17 11562604 9797758
/private/cyrus/config/mailboxes.db
lmtpd 27332 cyrus9u REG 8,170 11698179
/private/cyrus/config/socket/lmtp.lock
lmtpd 27332 cyrus 10u REG 8,17 4011 7017174
/private/cyrus/mail/stage./27332-1064247785
lmtpd 27332 cyrus 11u unix 0xce23fb00 55681436 socket
lmtpd 27332 cyrus 12u REG 8,17 589824 9797636
/private/cyrus/config/tls_sessions.db
lmtpd 27332 cyrus 13u REG 8,17 167 42107658
/private/cyrus/mail/k/user/krolickp/cyrus.header
I'm willing to let this problem stick around for a while if there is
something I can do to help debug this. It seems to be
Re: single quote in mailbox
On Tue, Sep 23, 2003 at 12:32:56PM -0400, Rob Siemborski wrote: > GOODCHARS is designed by default to be very conservative about what it > allows -- we could include a number of characters that we don't because > they are bad in various environemnts (especially the shell -- this > especially comes up when working with the mail store directly). > > I'm pretty sure there isn't any harm in including ', though I don't think > we'll be doing so in the base distribution. Could GOODCHARS be user-defined, in imapd.conf for example ? It could default to the current constant. IIRC, the RFC only forbid NUL and your hierarchy separator, so theorically any other character could be acceptable. I currently don't have the time nor the skill to implement that, but I may in the future if it is required for my current installation. -- Etienne GoyerLinux Québec Technologies Inc. http://www.LinuxQuebec.com [EMAIL PROTECTED]
Re: Cyrus IMAP use questions.
On Tue, 23 Sep 2003, Denis V. Suhanov wrote: > First of all, I wanted to implement incoming mail sorting via > procmail. Considering that none of my users have local shell accounts, > what is the right way to implementing individual procmail scripts for > those users who don't want to use sieve? Or should I use sieve as an > alternative? My only concern with it is that most of the clients my > users use are unaware of sieve and I have no clue how to provide them > an access to their sieve's scripts in this case. Unless your users are trying to implement a mailing list of some sort or do something very complicated, sieve should serve all of their needs. There's also a performance hit for using procmail -- the number of processes you need to fork to do a given mail delivery goes way up. timsieved implements the MANAGESIEVE protocol for which there are a number of web apps that can handle it (in addition, of course, to sieveshell). > And the last (and probably the most stupid) question is regarding > 'details delivery'. I know that it is possbile to specify the folder > name together with an e-mail address so the letter will be > automatically put in the folder specified ([EMAIL PROTECTED], for > example, is supposed to put an e-mail into 'work' folder give that it > exists). But I am not sure what parameters I have to specify in my > sendmail.mc to get this to work. I am using cyrusv2 local delivery (I > believe it uses LMTP, right?) Do I have to use any special rules or > pass any special parameters to the for this feature to work? As long as sendmail isn't stripping what is between the + and the @ this should be easy, just give "anyone" the "p" right on the folders in question. Then rjs3+foo will file into user.rjs3.foo -Rob -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Rob Siemborski * Andrew Systems Group * Cyert Hall 207 * 412-268-7456 Research Systems Programmer * /usr/contributed Gatekeeper
cyrdeliver permission
Hello, just wondered if there is a way to run cyrdeliver as another user than 'cyrus'? Thanks in advance, Wojtek -- Best regards, Wojtek mailto:[EMAIL PROTECTED]
Cyrus IMAP use questions.
Hello all, I've already asked a few questions here and the mailing list was extremely helpful (in fact, I only was able to start using Cyrus IMAP server after asking around here). Once again, I want to thank everybody who replied and helped me with my problems. Now I have a couple of questions regarding the neat little things I'd like to implement on my mail server upon user's requests. First of all, I wanted to implement incoming mail sorting via procmail. Considering that none of my users have local shell accounts, what is the right way to implementing individual procmail scripts for those users who don't want to use sieve? Or should I use sieve as an alternative? My only concern with it is that most of the clients my users use are unaware of sieve and I have no clue how to provide them an access to their sieve's scripts in this case. I also was asked to provide an ability to collect mail from other user's accounts automatically (mostly pop3) and to put it in my IMAP server's folders. Fetchmail seems to be useful for that, but the same problem appears - since my users do not have local accounts, I am not sure what would be a right way to create fetchmail's configuration files for each of them individually. I am thinking of providing something like an SSL-wrapped web page with authentication against sasldb for this, but still you might have a better idea on that. Speaking of this approach, is there a standard tool for this kind of authentication (SSL-ed webpage and sasldb)? I am asking because another problem I have here is letting users change their sasldb-based passwords without having to log in to the box. And the last (and probably the most stupid) question is regarding 'details delivery'. I know that it is possbile to specify the folder name together with an e-mail address so the letter will be automatically put in the folder specified ([EMAIL PROTECTED], for example, is supposed to put an e-mail into 'work' folder give that it exists). But I am not sure what parameters I have to specify in my sendmail.mc to get this to work. I am using cyrusv2 local delivery (I believe it uses LMTP, right?) Do I have to use any special rules or pass any special parameters to the for this feature to work? Thank you all very much, I would really appreciate any help with these problems. -- Sincerely yours, Denis mailto:[EMAIL PROTECTED]
Re: single quote in mailbox
On Tue, 23 Sep 2003, Patrick Welche wrote: > > I couldn't find any reason not to allow the single quote in the code or > > the rfcs either. > > It would be good to know what the reason was when that code was written.. > > > I was hoping somebody would answer this time... Although I can't tell > > you why the single quote is not in GOODCHARS, I can tell you that we > > added it in mboxname.c (as posted in my message) and we've been running > > in procuction for several months without any problems. > > Great - I will do the same. GOODCHARS is designed by default to be very conservative about what it allows -- we could include a number of characters that we don't because they are bad in various environemnts (especially the shell -- this especially comes up when working with the mail store directly). I'm pretty sure there isn't any harm in including ', though I don't think we'll be doing so in the base distribution. -Rob -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Rob Siemborski * Andrew Systems Group * Cyert Hall 207 * 412-268-7456 Research Systems Programmer * /usr/contributed Gatekeeper
cyrusv2 can´t deliver mail ....
Hello List. How i can´t tell to cyrusv2 mailer the complete RCPT address?. The message arrive to sendmail, sendmail detect the address as loca-host-name, stripped the host part and call cyrusv2 with only username. I don´t have any user into defualtdomain, all my users are or will be into /var/imap/domain structure. The imapd.log look like this. Sep 23 17:58:01 micalet lmtpunix[15824]: accepted connectionSep 23 17:58:01 micalet lmtpunix[15824]: lmtp connection preauth'd as postmanSep 23 17:58:01 micalet lmtpunix[15824]: append_check() of 'user.helio' failedSep 23 17:58:01 micalet master[15827]: about to exec /opt/cyrus/bin/lmtpdSep 23 17:58:01 micalet lmtpunix[15827]: executed I think that the rigth form could be: Sep 23 17:58:01 micalet lmtpunix[15824]: accepted connectionSep 23 17:58:01 micalet lmtpunix[15824]: lmtp connection preauth'd as postmanSep 23 17:58:01 micalet lmtpunix[15824]: append_check() of '[EMAIL PROTECTED]' OKSep 23 17:58:01 micalet master[15827]: about to exec /opt/cyrus/bin/lmtpdSep 23 17:58:01 micalet lmtpunix[15827]: executed Any hints? Thank in advanced Danny
Re: single quote in mailbox
On Tue, Sep 23, 2003 at 04:06:43PM +0100, Patrick Welche wrote: > On Tue, Sep 23, 2003 at 10:11:50AM -0400, Marc-Andre Gaudreau wrote: > > Hi Patrick, > > > > I pretty much asked the same question in this message : > > http://asg.web.cmu.edu/archive/message.php?mailbox=archive.info-cyrus&searchterm=gaudreau&msg=19930 PS Any chance of getting this committed? . create "user/brsr/colleges' commttee" . OK Completed . list "" * ... * LIST (\HasNoChildren) "/" "user/brsr/colleges' commttee" ... Cheers, Patrick
Re: single quote in mailbox
On Tue, Sep 23, 2003 at 10:11:50AM -0400, Marc-Andre Gaudreau wrote: > Hi Patrick, > > I pretty much asked the same question in this message : > http://asg.web.cmu.edu/archive/message.php?mailbox=archive.info-cyrus&searchterm=gaudreau&msg=19930 I'm sorry I missed it! I only came across a reference from a pine user trying to access cyrus :/ I suppose I should have done like you and just tried it! > I couldn't find any reason not to allow the single quote in the code or > the rfcs either. It would be good to know what the reason was when that code was written.. > I was hoping somebody would answer this time... Although I can't tell > you why the single quote is not in GOODCHARS, I can tell you that we > added it in mboxname.c (as posted in my message) and we've been running > in procuction for several months without any problems. Great - I will do the same. Cheers, Patrick
Re: single quote in mailbox
Hi Patrick, I pretty much asked the same question in this message : http://asg.web.cmu.edu/archive/message.php?mailbox=archive.info-cyrus&searchterm=gaudreau&msg=19930 I couldn't find any reason not to allow the single quote in the code or the rfcs either. I was hoping somebody would answer this time... Although I can't tell you why the single quote is not in GOODCHARS, I can tell you that we added it in mboxname.c (as posted in my message) and we've been running in procuction for several months without any problems. Patrick Welche wrote: As ever, I'm moving users from courier to cyrus. All was well until I hit a mailbox with a single quote in its name which cyrus rejects. I had a look at rfc3501 and as far as I can tell, it doesn't prohibit a ' (allows astring). imap/mboxname.c allows: #define GOODCHARS " +,-.0123456789:[EMAIL PROTECTED]" So, is there a reason for not allowing ', or is it a simple ;) matter of adding it to GOODCHARS and wondering about all the places in the C code which need \' ? Cheers, Patrick
append_check() of user.......
Hello list, Right now, i have a problem with sendmail 8.12.10 and cyrus-imap-2_2, the first one is accepting message but cyrus-imap is unable to deliver to the correct user inbox. My linux box is configured to use virtualdomains support. /etc/mail/local-host-name xxx.com yyy.com /etc/mail/mailertable xxx.com cyrusv2: [EMAIL PROTECTED] yyy.com cyrusv2: [EMAIL PROTECTED] The problem is that cyrusv2 is receiving only the address´s local part from the sendmail, i think that this could be cause the undelivery message with "user not found" reason. tail -f /var/log/imapd.log Sep 23 13:17:53 micalet lmtpunix[14257]: accepted connectionSep 23 13:17:53 micalet lmtpunix[14257]: lmtp connection preauth'd as postmanSep 23 13:17:53 micalet lmtpunix[14257]: append_check() of 'user.armando' failedSep 23 13:18:14 micalet master[13030]: process 14253 exited, status 0 Reading this log file, i saw this "user.armando" when the correct and validated user mailbox is [EMAIL PROTECTED]. How i can solve this problem ? Thank in advanced Danny
Cyrus 2.2-1 Beta Global Admin (cyrus) problem
hello, I have tried the Cyrus 2.2-1 Beta on RH8 and RH9. It is very strange that the line in /etc/imapd.conf : " admins: cyrus " does not behave the same. Since I use saslauthd with mech=LDAP, I have the record for [EMAIL PROTECTED] (!) because On RH8, it acts as the global admin which can modify all the users for different domains. It uses [EMAIL PROTECTED] as the global login(!). The log shows that the login name is [EMAIL PROTECTED] , even I just type " cyrus ". It just appends the @localdomain after the name?! On RH9, it is NOT the case. using the same LDAP, the [EMAIL PROTECTED] is the user(cyrus) for the domain, localdomin!? The log shows that the login name is [EMAIL PROTECTED] , even I just type " cyrus ". The cyrus admin has disappeared. And I cannot find a way to use cyrus admin again. Do anyone can clarify and provide me the guideline for this? Patrick Tsang
IGNORE: test
Just testing to see if I've cut out some of the lists with my anti-Gibe-F blocking :-) Nix.
Re: cyrus-imapd 2.1.15, sieve, lmtpd, and return-path header
On Tue, 23 Sep 2003, Pat Lashley wrote: > --On Monday, September 22, 2003 22:40:00 -0700 Chris Stromsoe > <[EMAIL PROTECTED]> wrote: > > I keep mentioning Envelope-To because 30+ years of Software Engineering > experience tells me that if you handle Return-Path, people will also > want Envelope-To. out of ~40,000 messages I've got handy, between 50 and 60 had some variant of envelope-to (either plain or with x- or old-), and 36,906 had a return-path header. Return-Path is an rfc mandated header that must be handled. I'm still not sure how Envelope-To applies to this discussion. > > That, imho, is not correct behavior. > > What headers, aside from Return-Path, are not cached in RAM? The Received header that lmtpd adds and the X-Sieve header. There may be others in other areas of the code, but those two and the Return-Path header are the three that I saw. > > I don't what the Sieve RFC says about return-path. I'm pretty sure > > that it isn't germane to this topic. > > Then why was it necessary to create an "envelope" extension to sieve? I don't know. Maybe early versions of lmtpd weren't following rfc2821 and properly settings the return-path, and sieve couldn't get at the information any other way. Maybe the envelope extension was added to get at envelope-to and envelope-from was added as an afterthought. > > smartsieve can't be smart enough to automatically translate the > > condition. It seems like you're suggesting that smartsieve (and any > > other program that generates sieve rules) should be following behind > > lmtpd, trying to guess at what special header might map to which > > special sieve function. > > Nonsense. I'm only suggesting that it be smart enough to actually > follow the RFCs and drafts; and to know about a few of the extensions > like envelope. There is no RFC or draft that says that the return-path header should not be visible as a header to sieve scripts, whereas 2821 does say that the final delivery must add the return-path header. To me, that says that return-path should be added by lmtpd and be visible to sieve. Making smartsieve -- and every other application out there that generates sieve scripts -- work around lmtpd's shortcomings doesn't make any sense to me. Fixing lmtpd so that the in-ram cache mirrors the on-disk data store does. > > I'm not sure what you mean here. Or maybe I am not being clear. To > > restate myself: this issue has _nothing to do with sieve_. I am not > > trying to patch sieve and I don't want to be. The patch changes the > > behavior of lmtpd so that the on-disk and in-memory represention of > > the headers is the same. (Rather, the patch is a start; since it > > doesn't yet include the last-hop Received header they don't completely > > match.) > > You're trying to change LMTP to provide a header that WILL NOT EXIST in > other sieve environments. It will not be at all obvious to the naive > user that that header was inserted at the last minute rather than > following the message from its inception. I still don't see what it is that you are trying to say. You seem to be assuming some sort of lmtp/sieve interdependance. Please correct me if I'm wrong. Other sieve environments may not have lmtp at all. sieve is just a set of tokens and rules for filtering. You could probably quite easily use sieve to parse nntp articles into mailboxes, or write a parser similar to procmail and run sieve out of sendmail as a delivery agent. As far as being not obvious to the naive user ... when has that stopped anything from happening? lmtpd rejects messages with 8-bit content, converts characters with no character-set defined to X, will bounce mail that has NULL characters embedded in it. The "naive users" are doing alright so far. > >> The envelope extension exists to provide the functionality you need. > > > > The envelope extension does not provide the functionality that I need. > > Open any message that is stored in cyrus and was delivered via lmtp. > > Look at the headers. The first header that you will see looks > > something like: > > > > Return-Path: <[EMAIL PROTECTED]> > > > > I need to be able to filter on that header. > > You are confusing functionality with implementation. A common mistake. No, I'm not: I need the functionality of being able to filter on all of the headers in the message. Return-Path is a header. lmtpd is the "final delivery" outside of the smtp environment. lmtpd adds the header on disk. It does not cache the header in the header cache. If there were a separate sieve parser (a la procmail) and lmtpd forked off sieve to take action, it would act on the on-disk file and would see the Return-Path header. The built-in sieve parser should see the same set of headers. -Chris > -Pat >
Re: cyrus-imapd 2.1.15, sieve, lmtpd, and return-path header
--On Monday, September 22, 2003 22:40:00 -0700 Chris Stromsoe <[EMAIL PROTECTED]> wrote: On Mon, 22 Sep 2003, Pat Lashley wrote: Return-Path doesn't exist at all until final delivery, at which point, I believe it is optional. from RFC2821 (which, I realize, is smtp and not lmtp) When the delivery SMTP server makes the "final delivery" of a message, it inserts a return-path line at the beginning of the mail data. This use of return-path is required; mail systems MUST support it. The return-path line preserves the information in the path from the MAIL command. Ok, I was wrong on that. It is still required. I'm not familiar with Envelope-To and have no idea what uses it or might use it. I don't really care about (and am not complaining about) any specific headers. Envelope-To does exactly what you would expect from the name. It records the recipient address from the envelope. It is completely non-standard; but quite common as there is no other method of saving that information with the message in a manner visible to the end user. (Return-Path would probably be named Envelope-Sender if it hadn't originally had a slightly different function back in the days of usenet bang-routed addresses.) I keep mentioning Envelope-To because 30+ years of Software Engineering experience tells me that if you handle Return-Path, people will also want Envelope-To. My problem is this: - a message is passed to lmtpd via lmtp - lmtpd writes the mail headers to disk and caches the headers in ram - lmtpd generates and writes a Return-Path header to disk but does not cache it in ram - lmtpd generates and writes other headers to disk but does not cache them in ram either - sieve operates by checking against the headers in the cache - sieve can't check all of the headers, because not all of them are in the cache That, imho, is not correct behavior. What headers, aside from Return-Path, are not cached in RAM? I don't what the Sieve RFC says about return-path. I'm pretty sure that it isn't germane to this topic. Then why was it necessary to create an "envelope" extension to sieve? smartsieve can't be smart enough to automatically translate the condition. It seems like you're suggesting that smartsieve (and any other program that generates sieve rules) should be following behind lmtpd, trying to guess at what special header might map to which special sieve function. Nonsense. I'm only suggesting that it be smart enough to actually follow the RFCs and drafts; and to know about a few of the extensions like envelope. > I'm not sure why the script won't work with any other conformant sieve > implementation. This has nothing to do with sieve... it has to do with > lmtpd writing one thing to disk and keeping a separate thing in ram. Because other sieve implementations are likely to be in an environment where that header does not yet exist to be tested. I'm not sure what you mean here. Or maybe I am not being clear. To restate myself: this issue has _nothing to do with sieve_. I am not trying to patch sieve and I don't want to be. The patch changes the behavior of lmtpd so that the on-disk and in-memory represention of the headers is the same. (Rather, the patch is a start; since it doesn't yet include the last-hop Received header they don't completely match.) You're trying to change LMTP to provide a header that WILL NOT EXIST in other sieve environments. It will not be at all obvious to the naive user that that header was inserted at the last minute rather than following the message from its inception. The envelope extension exists to provide the functionality you need. The envelope extension does not provide the functionality that I need. Open any message that is stored in cyrus and was delivered via lmtp. Look at the headers. The first header that you will see looks something like: Return-Path: <[EMAIL PROTECTED]> I need to be able to filter on that header. You are confusing functionality with implementation. A common mistake. -Pat
Re: cyrus-imapd 2.1.15, sieve, lmtpd, and return-path header
--On Tuesday, September 23, 2003 00:01:54 -0700 Andrew Morgan <[EMAIL PROTECTED]> wrote: It sounds to me like the return-path header should be added when the message is delivered to its final resting place. When sieve is looking at the message, it hasn't been delivered yet. A sieve script may decide to redirect the message or generate a bounce. In that case, the return-path should not be added to the message because it has not reached its final destination. When lmtpd writes the message to disk, it is *delivered*. Yes. Exactly. But where does return-path come from? Somehow lmtpd knows how to generate that header, but it's not clear to me how it does it. What I'm getting at is, maybe it is possible to filter on some other header that is equivalent to return-path. It gets it from the envelope. Hence the envelope extension to sieve; which allows testing not only of that value but of the envelope recipient as well. -Pat
Re: cyrus-imapd 2.1.15, sieve, lmtpd, and return-path header
On Tue, 23 Sep 2003, Andrew Morgan wrote: > I can't believe I'm wading into this discussion... :) :) > It sounds to me like the return-path header should be added when the > message is delivered to its final resting place. When sieve is looking > at the message, it hasn't been delivered yet. A sieve script may decide > to redirect the message or generate a bounce. In that case, the > return-path should not be added to the message because it has not > reached its final destination. > > When lmtpd writes the message to disk, it is *delivered*. Every message in the "stage." directory has a Return-Path header generated by lmtpd. When sieve rejects a message, it sends the file in the stage. directory back to the user with the Return-Path header (and the other generated headers) intact. Redirecting looks like it does the same thing (look at send_reject() and send_redirect() in lmtpd.c) -- it opens up sendmail and sends the body of message in the stage. directory. If the message on disk is the "delivered" message, the header cache should contain all of the same headers. > But where does return-path come from? Somehow lmtpd knows how to > generate that header, but it's not clear to me how it does it. > > What I'm getting at is, maybe it is possible to filter on some other > header that is equivalent to return-path. return-path is being built from the envelope from passed through in the lmtp dialogue. -Chris > Andy > >
Re: cyrus-imapd 2.1.15, sieve, lmtpd, and return-path header
On Mon, 22 Sep 2003, Chris Stromsoe wrote:
> On Mon, 22 Sep 2003, Pat Lashley wrote:
>
> > Return-Path doesn't exist at all until final delivery, at which point, I
> > believe it is optional.
>
> from RFC2821 (which, I realize, is smtp and not lmtp)
>
>When the delivery SMTP server makes the "final delivery" of a
>message, it inserts a return-path line at the beginning of the mail
>data. This use of return-path is required; mail systems MUST support
>it. The return-path line preserves the information in the
>path from the MAIL command.
I can't believe I'm wading into this discussion... :)
It sounds to me like the return-path header should be added when the
message is delivered to its final resting place. When sieve is looking at
the message, it hasn't been delivered yet. A sieve script may decide to
redirect the message or generate a bounce. In that case, the return-path
should not be added to the message because it has not reached its final
destination.
When lmtpd writes the message to disk, it is *delivered*.
> The envelope extension does not provide the functionality that I need.
> Open any message that is stored in cyrus and was delivered via lmtp.
> Look at the headers. The first header that you will see looks something
> like:
>
> Return-Path: <[EMAIL PROTECTED]>
>
> I need to be able to filter on that header. In my opinion, sieve code
> like
>
> if anyof (header :contains "return-path" "[EMAIL PROTECTED]") {
> fileinto "cbs";
> }
>
> needs to work
But where does return-path come from? Somehow lmtpd knows how to generate
that header, but it's not clear to me how it does it.
What I'm getting at is, maybe it is possible to filter on some other
header that is equivalent to return-path.
Andy
