Re: [sendmail] lmtp, cyrusv2d, shared folders and case
Andrzej Adam Filip a écrit : > nodens2099 wrote: > > >> I tried setting >> "lmtp_downcase_rcpt: 0" >> in imapd.conf, adding the u flag to the cyrusv2d mailer, but to no avail. >> >> the +part is always converted to lower case. >> > > Post your cyrusv2d definition from sendmail.cf. > echo '=M' | sendmail -bt | grep cyrusv2d > > Does it contain F=u flag? > > Yes, but only once I added the F=u flag to the m4 file (MODIFY_MAILER_FLAGS(`cyrusv2d', `+u') in the sendmail.mc didn't work). mailer 4 (cyrusv2d): P=[IPC] S=EnvFromSMTP/HdrFromL R=EnvToSMTP/EnvToSMTP M=0 U=-1:-1 F=1DFMXlmnqsuz L=0 E=\r\n T=DNS/RFC822/SMTP r=100 A=FILE /var/run/cyrus/socket/lmtp Thanks ! > By default sendmail's mailers convert user part of recipient address to > lowercase. F=u flag stops it. > *BUT* it would apply to "normal recipients" *too*. > > That whas my understanding as well. but it's ok, since lmtp_downcase_rcpt: yes takes care of the username part. >> initially I tried using the mrs_cyrus_mailertable approach but it was even >> worse, as checking +folder/subfol...@domaine.tld with smmap returns unknown >> user, even when the folder exists and has the "p" acl. >> > > Does smmap return "unknown user" *even for lowercase folders*? > > Nope, you're right. the real problem is on the mrs check, then. If I create a lowerrcase folder along the mixed cased one, the mail is received in the correct folder (that is, the mixed case). >> Any clue ? >> >> I'd really like to avoid converting all existing folders to lower case of >> possible, it would be painful. >> > > It merely a matter of choosing a simple way you like anyway :-) > > BTW > Why have not you tried RTCyrus3 recipe? > [ It would require small "patching" too to meet your requirements ] > > > Actually, I was adding a new backend in an existing murder environment, migrating existing users and public folders from MS Exchange server. Since rtcyrusv2 "just works" in the existing environment, I didn't even check if there was an update. I saw about RTCyrus3 only when searching a solution to my public folder problem. if it can resolve the mrs_cyrus_mailertable problem, and if it's not too difficult to upgrade rtcyrus2 with it, I'm willing to give it a try. Regards, -- Clément Hermann (nodens) Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: [sendmail] lmtp, cyrusv2d, shared folders and case
Andrzej Adam Filip a écrit : >>> Does smmap return "unknown user" *even for lowercase folders*? >>> >> Nope, you're right. the real problem is on the mrs check, then. > > It can use only what smmap provides. > Are you ready to ask for changed in Cyrus' smmap to make it capable to > check Public folders availability? > > mrs_cyrus_mailertable may be changed to check validity of mailbox folder > its own list of valid folders. > This is already what I'm doing with ldap. There is no need to add another database, which would have to be maintained as the public folder list change, IMO. I actually took a look at the smmapd code, and it uses the lmtp_downcase_rcpt value to know whether the recipient should be converted to downcase or not. According to the code, it convert the user part only, and has a special case for shared folder, so it should be ok. Do you know a way to test smmapd manually ? >> if it can resolve the mrs_cyrus_mailertable problem, and if it's not too >> difficult to upgrade rtcyrus2 with it, I'm willing to give it a try. > > It can be fixed but you should decide which path do you want do go: > a) Cyrus smmap can tell if public folders is accessible > b) sendmail uses its own list of valid publicly accessible Cyrus public > folders > the a) is definitely the way to go. -- Clément Hermann (nodens) Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: [sendmail] lmtp, cyrusv2d, shared folders and case
nodens2099 wrote: > Andrzej Adam Filip a écrit : >> nodens2099 wrote: >> >> >>> I tried setting >>> "lmtp_downcase_rcpt: 0" >>> in imapd.conf, adding the u flag to the cyrusv2d mailer, but to no avail. >>> >>> the +part is always converted to lower case. >>> >> >> Post your cyrusv2d definition from sendmail.cf. >> echo '=M' | sendmail -bt | grep cyrusv2d >> >> Does it contain F=u flag? > > Yes, but only once I added the F=u flag to the m4 file > (MODIFY_MAILER_FLAGS(`cyrusv2d', `+u') in the sendmail.mc didn't work). > > mailer 4 (cyrusv2d): P=[IPC] S=EnvFromSMTP/HdrFromL > R=EnvToSMTP/EnvToSMTP M=0 U=-1:-1 F=1DFMXlmnqsuz L=0 E=\r\n > T=DNS/RFC822/SMTP r=100 A=FILE /var/run/cyrus/socket/lmtp > > Thanks ! > >> By default sendmail's mailers convert user part of recipient address to >> lowercase. F=u flag stops it. >> *BUT* it would apply to "normal recipients" *too*. > > That whas my understanding as well. > but it's ok, since lmtp_downcase_rcpt: yes takes care of the username part. > >>> initially I tried using the mrs_cyrus_mailertable approach but it was even >>> worse, as checking +folder/subfol...@domaine.tld with smmap returns unknown >>> user, even when the folder exists and has the "p" acl. >> >> Does smmap return "unknown user" *even for lowercase folders*? >> > Nope, you're right. the real problem is on the mrs check, then. It can use only what smmap provides. Are you ready to ask for changed in Cyrus' smmap to make it capable to check Public folders availability? mrs_cyrus_mailertable may be changed to check validity of mailbox folder its own list of valid folders. > If I create a lowerrcase folder along the mixed cased one, the mail is > received in the correct folder (that is, the mixed case). > >>> Any clue ? >>> >>> I'd really like to avoid converting all existing folders to lower case of >>> possible, it would be painful. >>> >> >> It merely a matter of choosing a simple way you like anyway :-) >> >> BTW >> Why have not you tried RTCyrus3 recipe? >> [ It would require small "patching" too to meet your requirements ] > > Actually, I was adding a new backend in an existing murder environment, > migrating existing users and public folders from MS Exchange server. > Since rtcyrusv2 "just works" in the existing environment, I didn't even > check if there was an update. I saw about RTCyrus3 only when searching a > solution to my public folder problem. > > if it can resolve the mrs_cyrus_mailertable problem, and if it's not too > difficult to upgrade rtcyrus2 with it, I'm willing to give it a try. It can be fixed but you should decide which path do you want do go: a) Cyrus smmap can tell if public folders is accessible b) sendmail uses its own list of valid publicly accessible Cyrus public folders -- [pl>en: Andrew] Andrzej Adam Filip : a...@onet.eu She's the kind of girl who climbed the ladder of success wrong by wrong. -- Mae West Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Weekly/Monthly record-keeping / maintenance?
Okay, well, for what it's worth, I fixed all of the problems prohibiting me from running a quota -f to completion. The problem: imapdu.pl is buggy It fails to do the right thing with mailboxes containing a space in the name. ... 1.46 MB 114 msgs INBOX.BMP 1.46 MB 114 msgs INBOX.Bio Stuff 0.00 bytes 0 msgs INBOX.Drafts 1.25 MB 36 msgs INBOX.HLT 1.25 MB 36 msgs INBOX.Information Retrieval ... # $Id: imapdu.pl,v 1.9 2008/04/04 12:47:14 murch Exp $ I don't suppose 'murch', the author of the code reads this list? Jeff Blaine wrote: > Well, hmm... > > % sudo -u cyrus /local/mail/cyrus/bin/quota -f > user.ahe: usage was 148771194, now 148756681 > ... > user.william: usage was 107244707, now 107116597 > failed opening header for mailbox 'user.jay': System I/O error: %m > failed building quota list for '*': System I/O error: %m > > Apr 1 15:56:12 our.host.org quota[9819]: [ID 136705 local6.error] > IOERROR: opening /var/spool/imap/user/jay/cyrus.header: No such file or > directory > Apr 1 15:56:12 our.host.org quota[9819]: [ID 357877 local6.error] > failed opening header for mailbox 'user.jay': System I/O error: Bad file > number > Apr 1 15:56:13 our.host.org quota[9819]: [ID 809228 local6.error] > failed building quota list for '*': System I/O error: Bad file number > > Wesley Craig wrote: >> On 31 Mar 2009, at 16:50, Jeff Blaine wrote: >>> What I mean, is, for one example -- a user is currently >>> reporting that 'FETCH' (via the imapdu command) is showing >>> 142 messages totalling 640KB in a folder that is actually >>> completely empty on disk (except for cyrus.* files). >> Fetch doesn't examine the messages in the mailbox if it doesn't need >> to. Since you've reonstructed already, I wouldn't expect the seen DB is >> the problem. Probably the quota. Try rebuilding it. >> >> :wes >> > > Cyrus Home Page: http://cyrusimap.web.cmu.edu/ > Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki > List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html > Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: [sendmail] lmtp, cyrusv2d, shared folders and case
nodens2099 wrote: > Andrzej Adam Filip a écrit : Does smmap return "unknown user" *even for lowercase folders*? >>> Nope, you're right. the real problem is on the mrs check, then. >> >> It can use only what smmap provides. >> Are you ready to ask for changed in Cyrus' smmap to make it capable to >> check Public folders availability? >> >> mrs_cyrus_mailertable may be changed to check validity of mailbox folder >> its own list of valid folders. >> > This is already what I'm doing with ldap. There is no need to add > another database, which would have to be maintained as the public folder > list change, IMO. OK - smmap map be needless for "LDAP centric" configuration. > I actually took a look at the smmapd code, and it uses the > lmtp_downcase_rcpt value to know whether the recipient should be > converted to downcase or not. > According to the code, it convert the user part only, and has a special > case for shared folder, so it should be ok. > > Do you know a way to test smmapd manually ? There are simple socket map client and server perl scripts in contrib directory of sendmail distribution. >>> if it can resolve the mrs_cyrus_mailertable problem, and if it's not too >>> difficult to upgrade rtcyrus2 with it, I'm willing to give it a try. >> >> It can be fixed but you should decide which path do you want do go: >> a) Cyrus smmap can tell if public folders is accessible >> b) sendmail uses its own list of valid publicly accessible Cyrus public >> folders > > the a) is definitely the way to go. I "patch" only sendmail.cf :-) -- [pl>en: Andrew] Andrzej Adam Filip : a...@onet.eu Woman inspires us to great things, and prevents us from achieving them. -- Dumas Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Weekly/Monthly record-keeping / maintenance?
On Thu, Apr 02, 2009 at 10:42:23AM -0400, Jeff Blaine wrote:
> Okay, well, for what it's worth, I fixed all of the
> problems prohibiting me from running a quota -f
> to completion.
>
> The problem: imapdu.pl is buggy
Doh.
> It fails to do the right thing with mailboxes containing
> a space in the name.
Yeah, that's not entirely a surprise. Spaces in names
confuse lots of stuff.
> ...
> 1.46 MB 114 msgs INBOX.BMP
> 1.46 MB 114 msgs INBOX.Bio Stuff
> 0.00 bytes 0 msgs INBOX.Drafts
> 1.25 MB 36 msgs INBOX.HLT
> 1.25 MB 36 msgs INBOX.Information Retrieval
> ...
>
> # $Id: imapdu.pl,v 1.9 2008/04/04 12:47:14 murch Exp $
>
> I don't suppose 'murch', the author of the code reads
> this list?
Yeah, he's around, though he doesn't always see stuff as quickly
if it's only sent to the list. I've CC'd him.
That said, he's likely not the author - just the last person that
changed things. Given that he did a giant sweeping copyright
update of nearly every file in the tree a couple of months ago...
my ($rc, $msg) = $cyrus->send('', '', "EXAMINE $mb");
if ($rc eq 'OK') {
} else {
print "failed: $mb: $msg\n";
}
Apart from being icky perl, that will fail to change mailboxes
because the EXAMINE command will have dodgy syntax. I'm not
entirely sure why you're not seeing the 'failed' messages
though... The "$mb" needs to be at least quoted - which is
why I generally use something like Mail::IMAPTalk that can
do correct protocol quoting.
Ahh - it probably does output the "failed" stuff further up.
Bron.
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Weekly/Monthly record-keeping / maintenance?
I can guarantee that I had nothing to do with writing this code. Other
than updating the copyright blurb as Bron noted, I'm fairly certain I've
never even looked at it. Since its only example code, use it at your
own risk. If you fix it or improve it, feel free to pass us the changes.
Bron Gondwana wrote:
> On Thu, Apr 02, 2009 at 10:42:23AM -0400, Jeff Blaine wrote:
>> Okay, well, for what it's worth, I fixed all of the
>> problems prohibiting me from running a quota -f
>> to completion.
>>
>> The problem: imapdu.pl is buggy
>
> Doh.
>
>> It fails to do the right thing with mailboxes containing
>> a space in the name.
>
> Yeah, that's not entirely a surprise. Spaces in names
> confuse lots of stuff.
>
>> ...
>> 1.46 MB 114 msgs INBOX.BMP
>> 1.46 MB 114 msgs INBOX.Bio Stuff
>> 0.00 bytes 0 msgs INBOX.Drafts
>> 1.25 MB 36 msgs INBOX.HLT
>> 1.25 MB 36 msgs INBOX.Information Retrieval
>> ...
>>
>> # $Id: imapdu.pl,v 1.9 2008/04/04 12:47:14 murch Exp $
>>
>> I don't suppose 'murch', the author of the code reads
>> this list?
>
> Yeah, he's around, though he doesn't always see stuff as quickly
> if it's only sent to the list. I've CC'd him.
>
> That said, he's likely not the author - just the last person that
> changed things. Given that he did a giant sweeping copyright
> update of nearly every file in the tree a couple of months ago...
>
> my ($rc, $msg) = $cyrus->send('', '', "EXAMINE $mb");
> if ($rc eq 'OK') {
> } else {
> print "failed: $mb: $msg\n";
> }
>
> Apart from being icky perl, that will fail to change mailboxes
> because the EXAMINE command will have dodgy syntax. I'm not
> entirely sure why you're not seeing the 'failed' messages
> though... The "$mb" needs to be at least quoted - which is
> why I generally use something like Mail::IMAPTalk that can
> do correct protocol quoting.
>
> Ahh - it probably does output the "failed" stuff further up.
>
> Bron.
>
--
Kenneth Murchison
Systems Programmer
Carnegie Mellon University
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Weekly/Monthly record-keeping / maintenance?
I tried both
a) escaping spaces in $mb
b) quoting $mb
Neither worked.
That's about as far as I delve into Perl anymore, so
I'm happy to just retract my note to our users about
this tool.
Bron Gondwana wrote:
> On Thu, Apr 02, 2009 at 10:42:23AM -0400, Jeff Blaine wrote:
>> Okay, well, for what it's worth, I fixed all of the
>> problems prohibiting me from running a quota -f
>> to completion.
>>
>> The problem: imapdu.pl is buggy
>
> Doh.
>
>> It fails to do the right thing with mailboxes containing
>> a space in the name.
>
> Yeah, that's not entirely a surprise. Spaces in names
> confuse lots of stuff.
>
>> ...
>> 1.46 MB 114 msgs INBOX.BMP
>> 1.46 MB 114 msgs INBOX.Bio Stuff
>> 0.00 bytes 0 msgs INBOX.Drafts
>> 1.25 MB 36 msgs INBOX.HLT
>> 1.25 MB 36 msgs INBOX.Information Retrieval
>> ...
>>
>> # $Id: imapdu.pl,v 1.9 2008/04/04 12:47:14 murch Exp $
>>
>> I don't suppose 'murch', the author of the code reads
>> this list?
>
> Yeah, he's around, though he doesn't always see stuff as quickly
> if it's only sent to the list. I've CC'd him.
>
> That said, he's likely not the author - just the last person that
> changed things. Given that he did a giant sweeping copyright
> update of nearly every file in the tree a couple of months ago...
>
> my ($rc, $msg) = $cyrus->send('', '', "EXAMINE $mb");
> if ($rc eq 'OK') {
> } else {
> print "failed: $mb: $msg\n";
> }
>
> Apart from being icky perl, that will fail to change mailboxes
> because the EXAMINE command will have dodgy syntax. I'm not
> entirely sure why you're not seeing the 'failed' messages
> though... The "$mb" needs to be at least quoted - which is
> why I generally use something like Mail::IMAPTalk that can
> do correct protocol quoting.
>
> Ahh - it probably does output the "failed" stuff further up.
>
> Bron.
>
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Why do lmtpd processes accumulate?
We have a fairly conventional Cyrus server with one front-end and one back-end. Recently, I've noticed that when the number of lmtpd processes on the back-end server increases to the 400 range, performance drops to a crawl, including local deliveries. When I put an upper limit of 128 or 64 to these processes on the front-end, which requires a Cyrus restart, all of the local deliveries succeed in a short time. Performance also comes back to normal. I can't tell if it's the restart that fixes the problem or if it's the limit on lmtpd children. I'm wondering, though, if the lmtpd processes are all waiting on some Cyrus database, so that more of them just makes it worse. These are the databases, from imapd.conf: annotation_db: skiplist duplicate_db: berkeley-nosync mboxkey_db: skiplist mboxlist_db:skiplist quota_db: quotalegacy seenstate_db: skiplist subscription_db:flat tlscache_db:berkeley-nosync I believe those are current recommendations. Which ones might be causing the problem? Is there tuning that can be done on them? -- -Gary Mills--Unix Support--U of M Academic Computing and Networking- Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
