Re: [Dovecot] Please advise on very fast search
Hello, Stan, > This is why I recommended mbox in the first place. If your only writes > to these mailbox files are appends of new messages, mbox is the best > format by far. It's faster at appending than any other format, and it's > faster for searching than any other. I now seriously consider to use mdbox due to its nice self-regulation. After all it I believe mdbox should do file compression on its own, no cron scripts required. > It's an archive. You're not going to use maildir so you don't need > random IOPS performance. Thus RAID5/6 are a much better fit for an > archive as you get better read performance, with more than adequate > write performance, and you use less disks. And as this is an archive, > you don't need real time automatic/transparent compression. Thus I > recommend something like: > > 1. Debian 6 w/linux-image-2.6.39-bpo.2-amd64 or a custom rolled > 2.6.39 or later kernel > 2. hardware RAID5 w/large (2TB) SATA disks, 512B native sectors > e.g. MegaRAID SAS 9261-8i, 4 Seagate Constellation ES ST2000NM0011 > Specify a strip size of 256KB for the array > Perma set /sys/block/sdX/read_ahead_kb to 512 so you're reading > ahead 1024 sectors at a time instead of the default of 256. This > will speed up your searches quite a bit. > 3. XFS filesystem on the RAID device, created with mkfs.xfs defaults > 4. mbox w/zlib plugin. Compress daily files each night with a script > 5. You don't need LVM with a good RAID card (or with mdraid). This > controller can expand the RAID5 up to 8 drives (up to 32 drives max > using SAS expanders) We are considering to get HP DL180G6 server for 8 or 14 drives bays (base model price is somewhat equal, but additional drives adds up cost) with HP Smart Array P410 RAID controller (some servers are equipped with this controller by default) with 256 Mb battery-backed cache, but I'll check your suggestions! What memory size should I plan in the server? You're talking about AMD64 OS image, and 64-bit software are like to consume more memory that 32-bit, so looks like your talking about pretty huge RAM, and I don't believe it's necessary, or maybe I'm wrong? Problem is I have no experience with XFS and not sure I can tune it in the best way, so I'll go with mkfs.xfs defaults, I think. Hope we'll see Dovecot 2.1.x stable soon, as I'd like to use fts plugins and 2.1 handle that much better, but I don't like the idea of use unstable in production. Thank you for taking your time on my case, yours, Alexander
Re: [Dovecot] Deleting a mailbox and all its children
On Wed, 16 Nov 2011 03:15:02 +0200, Timo Sirainen wrote: > On 16.11.2011, at 3.04, Micah Anderson wrote: > > > I'm looking at scripting a mechanism to delete a mailbox, which is easy > > with 'doveadm mailbox delete' -- however it gets complicated when there > > are children of arbitrary depth, for example, I may have this: > > > > restored.daily1.INBOX > > restored.daily1.Sent > > restored.daily1.Mystuff > > restored.daily1.Mystuff.foo > > > > I would like to delete the 'restored' mailbox and all its children (and > > unsubscribe the folders), but the 'doveadm mailbox delete' command wont > > delete any children. > > > > I'm surprised that there isn't a -r (recursive) option to 'doveadm > > mailbox delete', but perhaps I am missing the easier way to handle this? > > Well, how about something like: > > doveadm mailbox list 'restored.*' | xargs -1 doveadm mailbox delete > doveadm mailbox delete restored That wont work because 'restored.daily1.Mystuff.foo' wont get removed before 'restored.daily1.Mystuff' is attempted, resulting in the following: doveadm mailbox list 'restored.*' | xargs -1 doveadm mailbox delete doveadm(micah): Error: Can't delete mailbox restored.daily1.Mystuff: Mailbox has children, delete them first The problem is the subfolders need to be removed first and then walk up the tree. perhaps I can do: doveadm mailbox list 'restored.*' | sort -r | xargs -n1 doveadm mailbox delete doveadm mailbox delete restored although I'm not sure if the reverse sort is going to work in all cases. micah pgpjj3dO5EJlx.pgp Description: PGP signature
Re: [Dovecot] Getting started with sieve and conversion from procmail
Hi, >> procmail: Assigning "DELIVER=/usr/libexec/dovecot/deliver" >> procmail: Executing "/usr/libexec/dovecot/deliver,-m,xspamtest" >> procmail: Non-zero exitcode (75) from "/usr/libexec/dovecot/deliver" >> procmail: Rescue of unfiltered data succeeded >> procmail: Executing "/usr/libexec/dovecot/deliver" >> procmail: Program failure (75) of "/usr/libexec/dovecot/deliver" >> procmail: Rescue of unfiltered data succeeded >> >> This was for a test user named "alexuser" I created. > > deliver should have logged something, or alternatively the problem was > exactly that it couldn't log anything. http://wiki.dovecot.org/LDA#logging Yes, looks like permissions. I have the following now in syslog: Nov 15 19:51:07 mail02 dovecot: lda(alexuser): Fatal: setresgid(12(mail),12(mail),12(mail)) failed with euid=500(alexuser): Operation not permitted Where is it trying to write that it can't? Does alexuser need to be a member of group mail? Thanks, Alex
Re: [Dovecot] Deleting a mailbox and all its children
On 16.11.2011, at 3.04, Micah Anderson wrote: > I'm looking at scripting a mechanism to delete a mailbox, which is easy > with 'doveadm mailbox delete' -- however it gets complicated when there > are children of arbitrary depth, for example, I may have this: > > restored.daily1.INBOX > restored.daily1.Sent > restored.daily1.Mystuff > restored.daily1.Mystuff.foo > > I would like to delete the 'restored' mailbox and all its children (and > unsubscribe the folders), but the 'doveadm mailbox delete' command wont > delete any children. > > I'm surprised that there isn't a -r (recursive) option to 'doveadm > mailbox delete', but perhaps I am missing the easier way to handle this? Well, how about something like: doveadm mailbox list 'restored.*' | xargs -1 doveadm mailbox delete doveadm mailbox delete restored That's what it would have to do internally anyway..
[Dovecot] Deleting a mailbox and all its children
I'm looking at scripting a mechanism to delete a mailbox, which is easy with 'doveadm mailbox delete' -- however it gets complicated when there are children of arbitrary depth, for example, I may have this: restored.daily1.INBOX restored.daily1.Sent restored.daily1.Mystuff restored.daily1.Mystuff.foo I would like to delete the 'restored' mailbox and all its children (and unsubscribe the folders), but the 'doveadm mailbox delete' command wont delete any children. I'm surprised that there isn't a -r (recursive) option to 'doveadm mailbox delete', but perhaps I am missing the easier way to handle this? thanks for any ideas or suggestions! micah -- pgpruGvZod8bb.pgp Description: PGP signature
Re: [Dovecot] Getting started with sieve and conversion from procmail
On 16.11.2011, at 2.08, Alex wrote: > No, I was just running as root from the command-line for an example. > The maillog shows error 75: > > procmail: Assigning "DELIVER=/usr/libexec/dovecot/deliver" > procmail: Executing "/usr/libexec/dovecot/deliver,-m,xspamtest" > procmail: Non-zero exitcode (75) from "/usr/libexec/dovecot/deliver" > procmail: Rescue of unfiltered data succeeded > procmail: Executing "/usr/libexec/dovecot/deliver" > procmail: Program failure (75) of "/usr/libexec/dovecot/deliver" > procmail: Rescue of unfiltered data succeeded > > This was for a test user named "alexuser" I created. deliver should have logged something, or alternatively the problem was exactly that it couldn't log anything. http://wiki.dovecot.org/LDA#logging
Re: [Dovecot] Getting started with sieve and conversion from procmail
Hi, >> It's now delivering, but my deliver command must be wrong. This is my >> ~/.procmailrc: > .. >> # # /usr/libexec/dovecot/deliver -m spamtest >> lda(root): Fatal: destination user parameter (-d user) not given >> >> Am I doing something wrong? > > Is it trying to deliver mails to root user? That's not supported. No, I was just running as root from the command-line for an example. The maillog shows error 75: procmail: Assigning "DELIVER=/usr/libexec/dovecot/deliver" procmail: Executing "/usr/libexec/dovecot/deliver,-m,xspamtest" procmail: Non-zero exitcode (75) from "/usr/libexec/dovecot/deliver" procmail: Rescue of unfiltered data succeeded procmail: Executing "/usr/libexec/dovecot/deliver" procmail: Program failure (75) of "/usr/libexec/dovecot/deliver" procmail: Rescue of unfiltered data succeeded This was for a test user named "alexuser" I created. Thanks again, Alex
Re: [Dovecot] Glued-together private namespaces
On Tue, Nov 15, 2011 at 11:39 AM, Timo Sirainen wrote:
> On Mon, 2011-11-14 at 10:23 -0800, Mark Moseley wrote:
>
>> > Thanks to a fortuitously unrelated thread ("how to disable quota for
>> > second namespace"), I got the quota part figured out and that seems to
>> > be working: Add a second entry to plugin {}, e.g. "quota2 =
>> > maildir:Archive quota:ns=INBOX.Archives." and add rules for
>> > userdb_quota2_rule, userdb_quota2_rule2, etc.
>> >
>> > My real question now is: Are there any fatal gotchas in this that I'm
>> > just not thinking of?
>> >
>>
>> Haven't had a chance to try this large-scale yet. Anybody have any
>> thoughts on it?
>
> The gotcha is that you have two completely independent quotas with
> independent usage/limits for the INBOX and Archive namespaces. If that
> is what you want, it should all be fine.
Nope, that's totally fine. The idea is to put Archive on cheaper
(slower) storage and then grant more generous quotas there to make it
worth their while to use, without slowing down their Inbox. Another
application would be to put their Spam in another namespace (for
people who choose to have it put in a separate folder) with a lower
quota, again to offload it onto cheaper storage, since hardly anyone
actually looks at it.
Or is this something that I could be doing more transparently in 2.1 with imapc?
Re: [Dovecot] Getting started with sieve and conversion from procmail
On 16.11.2011, at 2.00, Alex wrote: > It's now delivering, but my deliver command must be wrong. This is my > ~/.procmailrc: .. > # # /usr/libexec/dovecot/deliver -m spamtest > lda(root): Fatal: destination user parameter (-d user) not given > > Am I doing something wrong? Is it trying to deliver mails to root user? That's not supported.
Re: [Dovecot] Getting started with sieve and conversion from procmail
Hi, I just don't understand how to invoke procmail from deliver. >>> >>> The idea was to invoke deliver from procmail. I still do this myself >>> since I haven't bothered changing my procmail scripts to Sieve: >>> >>> DELIVER=/usr/local/libexec/dovecot/deliver >>> >>> :0 >>> * ^List-Id.*dovecot.dovecot.org >>> | $DELIVER -m dovecot >>> >>> :0 >>> | $DELIVER >> >> How do I tell dovecot to use procmail so that it may invoke deliver? > > You don't. You tell your MTA to invoke procmail, which is what I think you're > already doing? So the only change would be to modify your ~/.procmailrc. Okay, I'm making progress. I disabled the virtual_domain stuff in main.cf for postfix and just enabled mail_command=procmail. I think there's something wrong with the deliver command. It's now delivering, but my deliver command must be wrong. This is my ~/.procmailrc: # Log an abstract of all messages LOGABSTRACT=all PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/lib:/usr/local/bin LOGFILE=$HOME/.maillog MAILDIR=$HOME/mail PROCHOME=$HOME/.procmail SENDMAIL=/usr/sbin/sendmail VERBOSE=yes DELIVER=/usr/libexec/dovecot/deliver :0 fhW | $DELIVER -m xspamtest :0 fhw | $DELIVER When a mail is received, it prints this to .maillog: procmail: [23382] Tue Nov 15 18:56:04 2011 procmail: Assigning "COMSAT=no" procmail: Assigning "COMSAT=no" procmail: Assigning "DELIVER=/usr/libexec/dovecot/deliver" procmail: Executing "/usr/libexec/dovecot/deliver,-m,xspamtest" procmail: [23383] Tue Nov 15 18:56:04 2011 procmail: Assigning "COMSAT=no" procmail: Assigning "COMSAT=no" procmail: Assigning "DELIVER=/usr/libexec/dovecot/deliver" procmail: Executing "/usr/libexec/dovecot/deliver,-m,xspamtest" procmail: Non-zero exitcode (75) from "/usr/libexec/dovecot/deliver" procmail: Rescue of unfiltered data succeeded procmail: Executing "/usr/libexec/dovecot/deliver" procmail: Non-zero exitcode (75) from "/usr/libexec/dovecot/deliver" procmail: Rescue of unfiltered data succeeded procmail: Program failure (75) of "/usr/libexec/dovecot/deliver" procmail: Rescue of unfiltered data succeeded procmail: Locking "/var/spool/mail/alexuser.lock" procmail: Assigning "LASTFOLDER=/var/spool/mail/alexuser" procmail: Opening "/var/spool/mail/alexuser" procmail: Acquiring kernel-lock procmail: Executing "/usr/libexec/dovecot/deliver" procmail: Program failure (75) of "/usr/libexec/dovecot/deliver" procmail: Rescue of unfiltered data succeeded procmail: Locking "/var/spool/mail/alexuser.lock" procmail: Unlocking "/var/spool/mail/alexuser.lock" >From prvs=293a46ea3=m...@mydomain.com Tue Nov 15 18:56:04 2011 Subject: Grand Manor, Corvallis, OR, USA, AUTOMATED Extrane Folder: /var/spool/mail/bcc-user17025 It delivers to the mail spool, but executing deliver as its printed in the procmailrc: # # /usr/libexec/dovecot/deliver -m spamtest lda(root): Fatal: destination user parameter (-d user) not given Am I doing something wrong? Thanks, Alex
Re: [Dovecot] Excessive imap-logins, never die
On Wed, Nov 16, 2011 at 8:50 AM, Timo Sirainen wrote: > On 16.11.2011, at 0.48, Nick Edwards wrote: > > > Today I note there are 38 imap-logins that seem to be stale on our staff > > imap server, > > they do not seem to die off, new imap-logins are created with new logins > > and exit accordingly. > .. > > Dovecot version 1.2.17, built from source > > All this code has been rewritten in v2.0 and there's not much point in > spending time on figuring out why things aren't working perfectly in v1.x. > But you could most likely avoid this problem by switching to > "high-performance mode": http://wiki.dovecot.org/LoginProcess > > Okay, we do use high security mode specifically, and we require SSL on that server, I might just leave it as is, if there is no timeout value for these which I couldn't see when googling earlier today. I want to look at v2, but need to wait until all staff on are Christmas holidays, so if upgrade goes pair shaped, I can restore and not get bombarded with phone calls from management who can't get their mail.
Re: [Dovecot] Excessive imap-logins, never die
On 16.11.2011, at 0.48, Nick Edwards wrote: > Today I note there are 38 imap-logins that seem to be stale on our staff > imap server, > they do not seem to die off, new imap-logins are created with new logins > and exit accordingly. .. > Dovecot version 1.2.17, built from source All this code has been rewritten in v2.0 and there's not much point in spending time on figuring out why things aren't working perfectly in v1.x. But you could most likely avoid this problem by switching to "high-performance mode": http://wiki.dovecot.org/LoginProcess
[Dovecot] Excessive imap-logins, never die
Hiya, Today I note there are 38 imap-logins that seem to be stale on our staff imap server, they do not seem to die off, new imap-logins are created with new logins and exit accordingly. Now, these are from our monitoring software that runs on another server on our LAN. Our corporate link was taken down for 2 hours for urgent relocation of fibre (demolishing old adjoining building, and just in case, they needed to move it which we were glad about), so, this begs the question, why would imap-logins be there I guess in some sort of wait state, even with a still working internal DNS (internal view as well) the only difference is no actual internet connection in this time. A week ago when they first started to working and we lost internet for 45 minutes I also saw a few extra stale imap-logins, but after todays events, I decided this is a bigger issue that needs to be looked into, as dovecot required a restart to clear it all (I wasnt going to sit there killing individual PIDs all evening). Dovecot version 1.2.17, built from source
Re: [Dovecot] Getting started with sieve and conversion from procmail
On 16.11.2011, at 0.14, Alex wrote: >>> I just don't understand how to invoke procmail from deliver. >> >> The idea was to invoke deliver from procmail. I still do this myself >> since I haven't bothered changing my procmail scripts to Sieve: >> >> DELIVER=/usr/local/libexec/dovecot/deliver >> >> :0 >> * ^List-Id.*dovecot.dovecot.org >> | $DELIVER -m dovecot >> >> :0 >> | $DELIVER > > How do I tell dovecot to use procmail so that it may invoke deliver? You don't. You tell your MTA to invoke procmail, which is what I think you're already doing? So the only change would be to modify your ~/.procmailrc.
Re: [Dovecot] Getting started with sieve and conversion from procmail
Hi, >> I just don't understand how to invoke procmail from deliver. > > The idea was to invoke deliver from procmail. I still do this myself > since I haven't bothered changing my procmail scripts to Sieve: > > DELIVER=/usr/local/libexec/dovecot/deliver > > :0 > * ^List-Id.*dovecot.dovecot.org > | $DELIVER -m dovecot > > :0 > | $DELIVER How do I tell dovecot to use procmail so that it may invoke deliver? Thanks again, Alex
Re: [Dovecot] default Dovecot-LDA CHGRP/CHMOD
Thanks for the response. I was able to get it working with chmod 755 On Tue, Nov 15, 2011 at 3:34 PM, Timo Sirainen wrote: > On Wed, 2011-11-09 at 14:04 -0500, Chris Young wrote: > > I am familiar with this article > > http://wiki.dovecot.org/LDA#multipleuids > > > > I want to disable setuid-root, but I don't know the default group and > > permissions. > > > > sudo chgrp root /usr/libexec/dovecot/dovecot-lda > > sudo chmod 00750 /usr/libexec/dovecot/dovecot-lda > > > > but my delivery is still failing > > > > Nov 9 18:22:39 vl42 postfix/pipe[23039]: DEC54700B1: to=< > u...@domain.com>, > > orig_to=, relay=dovecot, delay=0.05, > delays=0.04/0/0/0.01, > > dsn=4.3.0, status=deferred (temporary failure. Command output: pipe: > fatal: > > pipe_command: execvp /usr/libexec/dovecot/dovecot-lda: Permission denied > ) > > dovecot-lda's group should be the same as you specify to the pipe in > Postfix's master.cf. > > >
Re: [Dovecot] dsync problems
On Tue, 2011-11-15 at 14:24 -0500, Micah Anderson wrote: > When a user renames their username, I am using dsync to copy their mail > over to the new username's mail location[0]. > > Some of the dsyncs are failing with errors that I dont know how to work > with, for example: > > dsync(): Error: Trying to open a non-listed mailbox with > guid=41fcd40303c8a64e43237ef44c7a > dsync(): Error: msg iteration failed: Couldn't open mailbox > 41fcd40303c8a64e43237ef44c7a > dsync(): Error: Trying to open a non-listed mailbox with > guid=41fcd40303c8a64e43237ef44c7a These shouldn't really happen. Something's going internally wrong with dsync. Can you reproduce this reliably somehow? > The errors cause a non-zero exit code from dsync, which causes my rename > script to bail out. What are these errors, and how can I fix them? Does a second dsync on error succeed? :) > 0. Why use dsync instead of a simple mv operation? This seems to be > necessary for two corner cases: > > 1. dovecot creates the new mailbox automatically when the user logs in > or receives a mail, so if the user changes their mail and logs in or > receives an email before the move has been done, then the mailbox is > created and then a move command will fail. > > 2. If there has been new mail created under the new name, we can't just > simply remove the stuff that is automatically created there and replace > it with the old things because we could potentially be removing mail > that has been delivered in the mean time. You could temporarily change the permissions for the home directory so that no new mailboxes/mails could be created during the move (e.g. 0700 root).
Re: [Dovecot] Seg fault in dovecot/auth 2.0.15
On Tue, 2011-11-15 at 14:31 -0500, Mark Goldey wrote: Could you do a bit more with gdb and this core: > #1 0x08059890 in auth_worker_handle_user (client=0x82b34d0) at > auth-worker-client.c:405 > auth_request = 0x82b56e8 > userdb_id = 1 fr 1 p *auth_request->userdb->userdb p *auth_request->userdb->userdb->iface
Re: [Dovecot] Dovecot SASL on another machine
On Sat, 2011-11-05 at 20:43 +0100, Patrick Ben Koetter wrote: > * Timo Sirainen : > > On Sat, 2011-11-05 at 15:13 +0100, Tobias Hachmer wrote: > > > > > is there a way to use Dovecot SASL Authentication for submission on a > > > different machine dovecot runs? > > > > With v2.0 you should be able to add inet_listener for auth service, > > which acts as an auth-client socket. > > > > > Dovecot machine <-> postfix machine(for submission) > > > > Looking at Postfix v2.8.0 code, looks like Postfix also supports it. > > Document it and Wietse will support it on the mailing list. If its not > documented support will be declined. You mean document in Dovecot wiki or for Postfix? > Should I? Please do :)
Re: [Dovecot] default Dovecot-LDA CHGRP/CHMOD
On Wed, 2011-11-09 at 14:04 -0500, Chris Young wrote: > I am familiar with this article > http://wiki.dovecot.org/LDA#multipleuids > > I want to disable setuid-root, but I don't know the default group and > permissions. > > sudo chgrp root /usr/libexec/dovecot/dovecot-lda > sudo chmod 00750 /usr/libexec/dovecot/dovecot-lda > > but my delivery is still failing > > Nov 9 18:22:39 vl42 postfix/pipe[23039]: DEC54700B1: to=, > orig_to=, relay=dovecot, delay=0.05, delays=0.04/0/0/0.01, > dsn=4.3.0, status=deferred (temporary failure. Command output: pipe: fatal: > pipe_command: execvp /usr/libexec/dovecot/dovecot-lda: Permission denied ) dovecot-lda's group should be the same as you specify to the pipe in Postfix's master.cf.
Re: [Dovecot] indexes permissions problem
On Thu, 2011-11-10 at 00:30 -0800, Mark Hanford wrote: > I've got a centos 6 server running Dovecot 2.0.beta6 (3156315704ef). > For legacy reasons (I'm moving mail from a Dovecot 1.1.1 and FreeBSD box > with user home directories NFS mounted), my index files are setup to be > in /u/indexes/ > > On the Dovecot 1.1.1 installation, the perms on the indexes directory is > 777 with root:mail ownership. > > The same thing on the Dovecot 2 / Centos server results in a 'permission > denied' error when Dovecot tries to create files. What exactly is the error message? Copy&paste it entirely. > So, I guess my main question is, what perms and ownership should > /u/indexes be set to? I've tried several different things before this > cry for help... Those permissions look ok to me. It's possible that this was a bug in 2.0.beta6 and upgrading might help (but might not).
Re: [Dovecot] Limit the number of pop/imap requests
On Thu, 2011-11-10 at 10:40 +0100, wwl wrote: > Hello Group, > > is it possible to Limit the number of pop/imap requests to Dovecot? > > I read the Wiki and found > >max_mail_processes = 512 > > is this the way to limit this ? This limits the number of simultaneous IMAP and POP3 connections. Is that what you mean with "requests"? If so, yeah, you can reduce this value. I think the counts are tracked separately for IMAP and POP3 (so you can have 512 IMAP and 512 POP3 connections).
Re: [Dovecot] Dovecot 2.0.15 quota configuration with mbox
On Thu, 2011-11-10 at 12:14 +0100, David Ocana wrote:
> >> namespace {
> >>separator = /
> >>prefix = INBOX/
> >>location =
> >> mbox:/var/empty:INBOX=/mail/%d/%n:INDEX=/var/dovecot/%d/%n
> >>inbox = yes
> >>hidden = yes
> >> }
> >>
> >> plugin {
> >>quota = dirsize:User quota
> >
> > quota = dirsize:User quota:ns=INBOX/
>
> Actually I forgot to mention that I also tried that, but I got the
> following error:
>
> Error: Initialization failed: Failed to initialize quota: Quota root
> User quota: Unknown parameter for backend dirsize: ns=INBOX/
Oh. Yeah, dirsize backend doesn't have all the features that other
backends have, because it's so little used. You could switch to dict
file quota..
Re: [Dovecot] dovecot-lda quota rule
On Thu, 2011-11-10 at 13:59 -0500, Micah Anderson wrote:
> I really like the feature where you can define quota rules with percents
> which trigger off of the default values[0] (so you can set the Trash to
> allow for 10% more of the user's quota for example).
>
> What I would really love in dovecot would be for the ability to
> configure a quota rule for dovecot-lda. I would like to configure things
> so we don't bounce emails for users until they are well over quota, the
> IMAP quota plugin is a really great way to notify people that they are
> over quota because it fails to write to other folders that should be
> enough to get people's attention that they need to deal with things, but
> bouncing is harsh.
You can give LDA-specific quota settings. Like maybe:
protocol lda {
plugin {
quota_rule2 = INBOX:storage=200%
}
}
or even not enforcing quota at all:
protocol lda {
plugin {
quota = maildir:User quota:noenforcing
}
}
Re: [Dovecot] Invalid "doveadm search" queries not rejected
On Thu, 2011-11-10 at 23:41 -0800, Peter Williams wrote: > I think I've encountered a bug in Dovecot 2.0.9 (as provided on CentOS > 6). > > According to the doveadm-search-query(7) manual page, a "mailbox" > specification can only come at the beginning of a search query. It doesn't exactly say that, but yeah, it doesn't also include MAILBOX and MAILBOX-GUID as search keys. Added now to future versions. > However, > if I put a mailbox specification in the middle of a query, not only do I > not get an error message, I get surprising results: > > $ doveadm search mailbox A |wc -l > 6979 > $ doveadm search mailbox B |wc -l > 206 > $ doveadm search mailbox A OR mailbox B |wc -l > 42492 I'm pretty sure this was fixed in v2.0.14. > If it became possible to do boolean logic on mailbox queries, that'd be > cool. It is.
Re: [Dovecot] Migration From Courier Imap.
On Sat, 2011-11-12 at 12:58 +0530, Rajeev Sharma wrote: > I am using vpopmail,qmail along with courier-imap 4.0 i want to > migrate from courier to dovecot but i have 100 shared folder > under /home/vpopmail/domains/domain.com/Malidir and user shared them > on the basis of rights assign. > > I am very much clear how to migrate from courier to dovecot but i am > unable how to migrate shared folder and assign rights. I don't think there is any existing documentation about this. I've also not used Courier shared folders myself much. I think the ACL files' contents are compatible, although Dovecot names them differently. IIRC Courier used some kind of symlinked maildir files for shared folders to provide per-user flags? Migrating those to Dovecot would be a bit tricky, especially because Dovecot currently supports only per-user \Seen flags (maybe Courier does too? I'm not sure) and Dovecot stores the per-user \Seen flags only in per-user index files. Anyway, I don't really have time to look into details of this, so you'll just need to figure out the specifics yourself (or alternatively pay me to do that via my company).
Re: [Dovecot] submission_host problem
On Sat, 2011-11-12 at 05:32 +0100, Bambero wrote: > I configured dovecot to use submission smtp host becouse of chroot. > > submission_host = 127.0.0.1 > > Unfortunatelly: > > Nov 12 05:11:15 myhost exim[23366]: 2011-11-12 05:11:15 SMTP protocol > synchronization error (next input sent too soon: pipelining was not > advertised): rejected "EHLO myhost" H=localhost [127.0.0.1] next > input="MAIL FROM:\\r\\n" This basically says that Dovecot's SMTP client is being noncompliant, because Exim doesn't advertise PIPELINING extension. But I'd rather not add more code to fix this, since PIPELINING gives you a little bit better performance anyway and you can most likely fix this by modifying Exim's configs in some way (I can't believe Exim wouldn't support PIPENING..).
Re: [Dovecot] Chroot
On Sun, 2011-11-13 at 17:25 +0100, Bambero wrote:
> all maildirs are located in: /var/mail/j...@example.com/Maildir
So this is /var/mail/%u/Maildir template.
> namespace {
> type = shared
> separator = /
> prefix = shared/%%u/
> location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u
You can replace this with:
location = maildir:/%%u/Maildir:INDEX=~/Maildir/shared/%%u
Although if this is used outside chroot (e.g. lmtp, doveadm) it won't
work correctly..
> Error: userdb lookup: connect(/var/dovecot/auth-userdb) failed: No
> such file or directory
This is done because you used %%h, to look up home directory from
userdb. If you don't use %%h, it's not done.
Re: [Dovecot] Glued-together private namespaces
On Mon, 2011-11-14 at 10:23 -0800, Mark Moseley wrote:
> > Thanks to a fortuitously unrelated thread ("how to disable quota for
> > second namespace"), I got the quota part figured out and that seems to
> > be working: Add a second entry to plugin {}, e.g. "quota2 =
> > maildir:Archive quota:ns=INBOX.Archives." and add rules for
> > userdb_quota2_rule, userdb_quota2_rule2, etc.
> >
> > My real question now is: Are there any fatal gotchas in this that I'm
> > just not thinking of?
> >
>
> Haven't had a chance to try this large-scale yet. Anybody have any
> thoughts on it?
The gotcha is that you have two completely independent quotas with
independent usage/limits for the INBOX and Archive namespaces. If that
is what you want, it should all be fine.
Re: [Dovecot] Getting started with sieve and conversion from procmail
On Tue, 2011-11-15 at 13:09 -0500, Alex wrote: > > > Yes, you can pipe the message to the Dovecot LDA from procmail. This would > > be similar to the solution described above, only using Procmail instead of a > > shell script. > > Are you referring to doing something like this: > > http://wiki2.dovecot.org/procmail > > I just don't understand how to invoke procmail from deliver. The idea was to invoke deliver from procmail. I still do this myself since I haven't bothered changing my procmail scripts to Sieve: DELIVER=/usr/local/libexec/dovecot/deliver :0 * ^List-Id.*dovecot.dovecot.org | $DELIVER -m dovecot :0 | $DELIVER etc.
[Dovecot] Seg fault in dovecot/auth 2.0.15
Folks:
I've been running Dovecot 2.0.7 on a Qmail / Vpopmail installation built
on Debian Squeeze, with no problems for about a year. Dovecot is used
for IMAP/S auth only.
Dovecot is built from source. I just upgraded to 2.0.15 but it
segfaults whenever an IMAP auth is attempted, regardless of the user who
tries to login, whether by webmail (Roundmail) Android phone (K-9 Mail)
or iPad (Roundmail)
I downgraded to 2.0.13 but the segfault persisted. Re-installed 2.0.7
and that cleared it right up.
Here's the pertinent details from 2.0.15.
Anything else I can provide?
root@XXX:/# uname -a
Linux XXX 2.6.32-5-686 #1 SMP Wed Jan 12 04:01:41 UTC 2011 i686 GNU/Linux
root@XXX:/# dovecot -n
# 2.0.15: /usr/local/etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-5-686 i686 Debian 6.0.3
auth_debug = yes
auth_debug_passwords = yes
auth_verbose = yes
auth_verbose_passwords = plain
disable_plaintext_auth = no
first_valid_uid =
last_valid_uid =
log_path = /var/log/dovecot.log
mail_debug = yes
passdb {
driver = vpopmail
}
protocols = imap
service imap-login {
inet_listener imap {
address = *
port = 143
}
inet_listener imaps {
address = *
port = 993
}
process_limit = 128
vsz_limit = 64 M
}
ssl_cert = , method=PLAIN,
rip=174.79.xxx.xxx, lip=192.xxx.xxx.xxx, mpid=691, TLS
Nov 15 14:08:43 imap-login: Warning: SSL alert: where=0x4008, ret=256:
warning close notify [174.79.xxx.xxx]
Nov 15 14:08:43 auth-worker: Debug: Loading modules from directory:
/usr/local/lib/dovecot/auth
I ought to mention that /usr/local/lib/dovecot/auth is empty.
Here's the backtrace:
root@XXX:/# gdb /usr/local/libexec/dovecot/auth
/usr/local/var/run/dovecot/core
GNU gdb (GDB) 7.0.1-debian
[snip]
Reading symbols from /usr/local/libexec/dovecot/auth...done.
warning: Can't read pathname for load map: Input/output error.
Reading symbols from /usr/local/lib/dovecot/libdovecot.so.0...done.
Loaded symbols for /usr/local/lib/dovecot/libdovecot.so.0
Reading symbols from /lib/i686/cmov/libcrypt.so.1...Reading symbols from
/usr/lib/debug/lib/i686/cmov/libcrypt-2.11.2.so...done.
(no debugging symbols found)...done.
Loaded symbols for /lib/i686/cmov/libcrypt.so.1
Reading symbols from /home/vpopmail/lib/libvpopmail.so...(no debugging
symbols found)...done.
Loaded symbols for /home/vpopmail/lib/libvpopmail.so
Reading symbols from /lib/i686/cmov/libdl.so.2...Reading symbols from
/usr/lib/debug/lib/i686/cmov/libdl-2.11.2.so...done.
(no debugging symbols found)...done.
Loaded symbols for /lib/i686/cmov/libdl.so.2
Reading symbols from /lib/i686/cmov/librt.so.1...Reading symbols from
/usr/lib/debug/lib/i686/cmov/librt-2.11.2.so...done.
(no debugging symbols found)...done.
Loaded symbols for /lib/i686/cmov/librt.so.1
Reading symbols from /lib/i686/cmov/libc.so.6...Reading symbols from
/usr/lib/debug/lib/i686/cmov/libc-2.11.2.so...done.
(no debugging symbols found)...done.
Loaded symbols for /lib/i686/cmov/libc.so.6
Reading symbols from /lib/ld-linux.so.2...Reading symbols from
/usr/lib/debug/lib/ld-2.11.2.so...done.
(no debugging symbols found)...done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /lib/i686/cmov/libpthread.so.0...Reading symbols
from /usr/lib/debug/lib/i686/cmov/libpthread-2.11.2.so...done.
(no debugging symbols found)...done.
Loaded symbols for /lib/i686/cmov/libpthread.so.0
Reading symbols from /home/vpopmail/etc/vpopmail.authmodule...(no
debugging symbols found)...done.
Loaded symbols for /home/vpopmail/etc/vpopmail.authmodule
Core was generated by `dovecot/auth -w'.
Program terminated with signal 11, Segmentation fault.
#0 0x in ?? ()
(gdb) bt full
#0 0x in ?? ()
No symbol table info available.
#1 0x08059890 in auth_worker_handle_user (client=0x82b34d0) at
auth-worker-client.c:405
auth_request = 0x82b56e8
userdb_id = 1
#2 auth_worker_handle_line (client=0x82b34d0) at auth-worker-client.c:534
id = 2
#3 auth_worker_input (client=0x82b34d0) at auth-worker-client.c:614
_data_stack_cur_id = 3
line =
ret = 232
#4 0xb785f021 in io_loop_call_io (io=0x82b3688) at ioloop.c:384
ioloop = 0x82a6398
t_id = 2
#5 0xb78600d3 in io_loop_handler_run (ioloop=0x82a6398) at
ioloop-epoll.c:213
ctx = 0x82b1f38
event = 0x82b1f78
list = 0x82b36b0
io = 0x829e2b0
tv = {tv_sec = 2147483, tv_usec = 0}
msecs = 1
ret = 1
i = 0
j = 0
call = 192
#6 0xb785efa0 in io_loop_run (ioloop=0x82a6398) at ioloop.c:405
No locals.
#7 0xb784abaa in master_service_run (service=0x82a62e8,
callback=0x805c470 ) at master-service.c:481
No locals.
#8 0x0805c828 in main (argc=2, argv=0x82a61c0) at main.c:298
c =
(gdb)
--mhg
Re: [Dovecot] getenv() in plugin not working
On Tue, 2011-11-15 at 03:58 +, Trutwin, Joshua wrote: > Appears this is the problem - I found this for dovecot v2 and it seems to be > working fine: > > http://sourceforge.jp/projects/dovecot2-drac/ > > Might wanna make a note on the wiki page that the DRAC setup is only for > dovecot 1.x? I changed the wiki2 link to this URL.
Re: [Dovecot] Question about pop3_reuse_xuidl
On Tue, 2011-11-15 at 16:02 +0100, Urban Loesch wrote: > At the beginning of our migration we didn't set the "pop3_reuse_xuidl" > configuration option to yes. > Do you know what happens when we activate the "pop3_reuse_xuidl" option on > our running dovecot? > > Will Dovecot change the uidl value for all existing mails which have set > "X-UIDL" in its mailheader, Yes. > or does Dovecot only change the uidl value for new received and still not > downloaded mails? Nope.. There is pop3_save_uidl=yes setting, which would do this for Maildir, assuming the POP3 clients had logged in while this option was enabled, but this isn't implemented for dbox. The biggest problem with implementing this for dbox would be that it has no variable length permanent per-mail storage (Maildir's dovecot-uidlist equivalent). One way to solve this would be by patching dsync a little bit, and then dsyncing existing mdbox to mdbox-new, which adds existing POP3 UIDLs as dbox metadata. Those will override X-UIDL: headers. The patching would be basically by making dsync/dsync-local-worker.c local_worker_msg_get_next() always set data.pop3_uidl variable to the correct UIDL (which would depend on your pop3_uidl_format setting - pop3/pop3-commands.c:pop3_get_uid() shows how).
[Dovecot] dsync problems
When a user renames their username, I am using dsync to copy their mail over to the new username's mail location[0]. Some of the dsyncs are failing with errors that I dont know how to work with, for example: dsync(): Error: Trying to open a non-listed mailbox with guid=41fcd40303c8a64e43237ef44c7a dsync(): Error: msg iteration failed: Couldn't open mailbox 41fcd40303c8a64e43237ef44c7a dsync(): Error: Trying to open a non-listed mailbox with guid=41fcd40303c8a64e43237ef44c7a dsync(): Error: Trying to open a non-listed mailbox with guid=42fcd40303c8a64e43237ef44c7a dsync(): Error: Trying to open a non-listed mailbox with guid=43fcd40303c8a64e43237ef44c7a dsync(): Error: Trying to open a non-listed mailbox with guid=44fcd40303c8a64e43237ef44c7a ERROR: dsync failed, exit code: 256 The errors cause a non-zero exit code from dsync, which causes my rename script to bail out. What are these errors, and how can I fix them? Thanks, micah 0. Why use dsync instead of a simple mv operation? This seems to be necessary for two corner cases: 1. dovecot creates the new mailbox automatically when the user logs in or receives a mail, so if the user changes their mail and logs in or receives an email before the move has been done, then the mailbox is created and then a move command will fail. 2. If there has been new mail created under the new name, we can't just simply remove the stuff that is automatically created there and replace it with the old things because we could potentially be removing mail that has been delivered in the mean time. I'd be really interested if people had suggestions for a better mechanism, or perhaps a way to have dovecot not create the new mail location automatically. -- pgp54e2yahexo.pgp Description: PGP signature
Re: [Dovecot] Please advise on very fast search
On Tue, 2011-11-15 at 12:26 -0600, Stan Hoeppner wrote: > This is why I recommended mbox in the first place. If your only writes > to these mailbox files are appends of new messages, mbox is the best > format by far. It's faster at appending than any other format, and it's > faster for searching than any other. Just as long as you're not simultaneously trying to read and write the mbox file (or just write in 2+ sessions). Then there's a lot waiting on locks. (mdbox has no read locks, and its write locks are very short lived.)
Re: [Dovecot] Please advise on very fast search
On 11/14/2011 3:16 PM, Alexander Chekalin wrote: > Locking issues on mbox is the reason for my long-lasting love affair with > maildir, Same reason most others fell in love with it. Many now want to divorce maildir, as the cost of the storage to maintain acceptable performance is now too high. > and it's lasts long years. Ok, the life's lessons are like this, learn > something and move on with it ;) even if it's "new old thing". Thank you for > pointing that! Many old UNIX gurus still use mbox, not maildir, and never will. If you ask them why they'll likely say "you don't use a screwdriver to drive a nail do you?" > What I was doubt about is default rotate size of 2M, since I used to see > pretty reasonable default settings in all Dovecot config. 32 or 64 are much > close to the ones I'd personally prefer. Given the fact that we're talking about an archive server, you'd be better off using a very large mdbox file size, say 1GB. You're never deleting individual messages from this archive correct? No expunges? This is why I recommended mbox in the first place. If your only writes to these mailbox files are appends of new messages, mbox is the best format by far. It's faster at appending than any other format, and it's faster for searching than any other. > I also about to choose now is the OS and FS for the archive. I seriously > think about ZFS with compression (in fact it will be stripes over couple of > mirrors = software equivalent of RAID 10 on SATA drives, with compression on > FS level) on FreeBSD, or XFS over LVM on Debian with compression in mdbox > itself. I see pros and contras for both, so that's the question to answer! It's an archive. You're not going to use maildir so you don't need random IOPS performance. Thus RAID5/6 are a much better fit for an archive as you get better read performance, with more than adequate write performance, and you use less disks. And as this is an archive, you don't need real time automatic/transparent compression. Thus I recommend something like: 1. Debian 6 w/linux-image-2.6.39-bpo.2-amd64 or a custom rolled 2.6.39 or later kernel 2. hardware RAID5 w/large (2TB) SATA disks, 512B native sectors e.g. MegaRAID SAS 9261-8i, 4 Seagate Constellation ES ST2000NM0011 Specify a strip size of 256KB for the array Perma set /sys/block/sdX/read_ahead_kb to 512 so you're reading ahead 1024 sectors at a time instead of the default of 256. This will speed up your searches quite a bit. 3. XFS filesystem on the RAID device, created with mkfs.xfs defaults 4. mbox w/zlib plugin. Compress daily files each night with a script 5. You don't need LVM with a good RAID card (or with mdraid). This controller can expand the RAID5 up to 8 drives (up to 32 drives max using SAS expanders) -- Stan
Re: [Dovecot] Getting started with sieve and conversion from procmail
Hi,
>>> The Sieve language doesn't support invoking external programs. You can
>>> execute this program before Sieve, for instance by wrapping the LDA
>>> invocation.
>>
>> Can you describe how I might do this?
>
> You can write a shell script that is executed from your MTA instead of the
> Dovecot LDA. Perform the process-mail task from within that script and then
> call the Dovecot LDA from the shell script (and pipe the message to it).
So you mean from master.cf, in the same way as deliver is being executed?
dovecot unix - n n - - pipe
flags=DRhu user=dovecot:dovecot argv=/usr/libexec/dovecot/deliver -f
${sender} -d ${recipient} -a "${RECIPIENT}"
Replacing deliver here with a shell script? Then execute deliver from
within the shell script, or just do the delivery from within a shell
script altogether?
>> Is there a way to just continue to use procmail?
>
> Yes, you can pipe the message to the Dovecot LDA from procmail. This would
> be similar to the solution described above, only using Procmail instead of a
> shell script.
Are you referring to doing something like this:
http://wiki2.dovecot.org/procmail
I just don't understand how to invoke procmail from deliver.
My main motivation for trying to continue to use procmail is that the
sieve language seems far more complicated and involved than even a
procmail recipe, and isn't as fully developed.
I don't want to make this even more complicated, however. Perhaps you
can explain how to filter a basic message based on something in the
body or a header and sort it into a file?
Thanks so much for your help.
Best,
Alex
Re: [Dovecot] Getting started with sieve and conversion from procmail
Op 15-11-2011 16:38, Alex schreef: Hi, The Sieve language doesn't support invoking external programs. You can execute this program before Sieve, for instance by wrapping the LDA invocation. Can you describe how I might do this? You can write a shell script that is executed from your MTA instead of the Dovecot LDA. Perform the process-mail task from within that script and then call the Dovecot LDA from the shell script (and pipe the message to it). Also, there is a non-standard (an still pretty much experimental) language extension for this, as provided by the pipe plugin: Is there a way to just continue to use procmail? Yes, you can pipe the message to the Dovecot LDA from procmail. This would be similar to the solution described above, only using Procmail instead of a shell script. Regards, Stephan.
Re: [Dovecot] Getting started with sieve and conversion from procmail
Hi,
>> I have a number of procmail scripts and I've converted them to sieve,
>> but they didn't convert very well. Can someone show me how to convert
>> the following:
>>
>> :0
>> {
>> :0 fhW
>> | /home/alex/bin/process-mail.pl
>> }
>>
>> The script skipped right over that.
>
> The Sieve language doesn't support invoking external programs. You can
> execute this program before Sieve, for instance by wrapping the LDA
> invocation.
Can you describe how I might do this?
> Also, there is a non-standard (an still pretty much
> experimental) language extension for this, as provided by the pipe plugin:
Is there a way to just continue to use procmail?
>> I'm also trying to convert the following:
>>
>> :0
>> * (RCVD_IN_DNSWL_HI|RCVD_IN_DNSWL_LOW|RCVD_IN_HOSTKARMA_W)
>> * !USER_IN_WHITELIST
>> * !BAYES_00
>> * !URIBL_
>> * !KHOP_RCVD_UNTRUST
>> * !RCVD_IN_DNSWL_MED
>> * !SPF_SOFTFAIL
>> * SPF_HELO_PASS
>> * SPF_PASS
>> {
>> LOG="DNSWL: "
>> :0:
>> whitelist/dnswl
>> }
>>
>> The script only produced the following:
>
> [...]
>>
>> Any ideas greatly appreciated.
>
> I'm not sure what these rules are supposed to mean (most conditions are
> obscured in variables), but by the looks of things these should also be
> executed outside Sieve as well. If all those conditions map to simple header
> and envelope tests, you can provide a Sieve alternative. However, I think
> USER_IN_WHITELIST queries some external list and that too is not (currently)
> possible from Pigeonhole Sieve. Also, it is not possible to change the log
> prefix from Sieve (as suggested by LOG=).
No, these are all simple header matches. There's no external program
being executed here. The LOG= is also a procmail built-in, but I can
do without that.
Thanks again,
Alex
[Dovecot] Question about pop3_reuse_xuidl
Hi, we are in migration progress from Communigate Pro 5.0.x to Dovecot 2.0.15 with mdbox. We still migrated about 25.000 IMAP accounts from CGP to Dovecot. Also there were still added about 2000 new POP3 Accounts on Dovecot. Now we must migrate about 10.000 POP3 accounts from CGP to Dovecot. At the beginning of our migration we didn't set the "pop3_reuse_xuidl" configuration option to yes. Do you know what happens when we activate the "pop3_reuse_xuidl" option on our running dovecot? Will Dovecot change the uidl value for all existing mails which have set "X-UIDL" in its mailheader, or does Dovecot only change the uidl value for new received and still not downloaded mails? Many thanks and regards Urban Loesch
Re: [Dovecot] getenv() in plugin not working
On 2011-11-14 10:58 PM, Trutwin, Joshua wrote: Might wanna make a note on the wiki page that the DRAC setup is only for dovecot 1.x? Why? You were on the wiki for v1, so obviously it is (or may be) only meant for v1. -- Best regards, Charles
