Re: [Dovecot] Dovecot 2.2.4 : Panic: file pop3-client.c: line 585 (client_default_destroy): assertion failed: (client->cmd == NULL)

[Ken A]

Thanks! I'll let you know if the issue disappears.
Ken Anderson


On 7/9/2013 9:20 PM, Timo Sirainen wrote:
> On 5.7.2013, at 21.02, Ken A  wrote:
> 
>> This doesn't seem to affect clients, but I'm seeing an occasional pop3
>> error in the log over the last week.
>> Any ideas what is causing this?
> ..
>>> Jul  4 15:04:03 mail dovecot: POP3(user)x.x.x.x: Connection
>>> closed top=0/0, retr=0/0, del=0/4582, size=246709805
>>
>>> Jul  4 15:04:03 mail dovecot: POP3(user)x.x.x.x: Panic: file
>>> pop3-client.c: line 585 (client_default_destroy): assertion failed:
>>> (client->cmd == NULL)
> 
> The only reason I can see for this is: 
> http://hg.dovecot.org/dovecot-2.2/rev/debbcfe5577a
> 
> 
> 

-- 
Ken Anderson
Pacific Internet - http://www.pacific.net

[Dovecot] Dovecot 2.2.4 : Panic: file pop3-client.c: line 585 (client_default_destroy): assertion failed: (client->cmd == NULL)

[Ken A]

This doesn't seem to affect clients, but I'm seeing an occasional pop3
error in the log over the last week.
Any ideas what is causing this?
Thanks,
Ken

> Jul  4 15:04:03 mail dovecot: POP3(user)x.x.x.x: Connection
> closed top=0/0, retr=0/0, del=0/4582, size=246709805

> Jul  4 15:04:03 mail dovecot: POP3(user)x.x.x.x: Panic: file
> pop3-client.c: line 585 (client_default_destroy): assertion failed:
> (client->cmd == NULL)

> Jul  4 15:04:03 mail dovecot: POP3(user)x.x.x.x: Error: Raw
> backtrace: /usr/lib/dovecot/libdovecot.so.0() [0x31b4c6074a] ->
> /usr/lib/dovecot/libdovecot.so.0() [0x31b4c60796] ->
> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x31b4c202df] ->
> dovecot/pop3() [0x404df9] -> dovecot/pop3(client_handle_input+0xd4)
> [0x4051b4] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36)
> [0x31b4c6f6d6] ->
> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xa7)
> [0x31b4c70737] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x28)
> [0x31b4c6f678] ->
> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13)
> [0x31b4c24fa3] -> dovecot/pop3(main+0x257) [0x4043d7] ->
> /lib64/libc.so.6(__libc_start_main+0xfd) [0x3a08e1ecdd] ->
> dovecot/pop3() [0x403f29]

> Jul  4 15:04:04 mail dovecot: POP3(user)x.x.x.x: Fatal: master:
> service(pop3): child 9080 killed with signal 6 (core dumped)


-- 
Ken Anderson
Pacific Internet - http://www.pacific.net

Re: [Dovecot] flat file in tmpfs for dict quota

[Ken A]

On 7/2/2013 8:48 AM, Steffen Kaiser wrote:
> On Thu, 27 Jun 2013, Ken A wrote:
> 
>> I'm using dict quota like so:
> 
>> quota = dict:User quota::file:/[path]/quotas/%u
> 
>> [path]/quotas/ is a tmpfs.
> 
>> The idea is to do less work on disk. Other than forcing dovecot
>> to rebuild quotas on a reboot, are there any downsides?
> 
> I would say no, but to recalc the quota file might be more
> difficult that you think, make sure no logins or deliveries or
> automatic scripts change the content of the mail storage.

The quota files are per mailbox, and are created when the user logs in
or LDA touches a mailbox. Most mailboxes are < 1GB. Dovecot seems to
handle it very quickly.
Thanks,
Ken Anderson

> Regards,
> 
> -- Steffen Kaiser
> 

-- 
Ken Anderson
Pacific Internet - http://www.pacific.net

[Dovecot] flat file in tmpfs for dict quota

[Ken A]

I'm using dict quota like so:

quota = dict:User quota::file:/[path]/quotas/%u

[path]/quotas/ is a tmpfs.

The idea is to do less work on disk. Other than forcing dovecot to
rebuild quotas on a reboot, are there any downsides?

Thanks,
Ken

-- 
Ken Anderson
Pacific Internet - http://www.pacific.net

Re: [Dovecot] Warning recipient when message delivery fails due to over-quota

[Ken A]

On 6/24/2013 1:02 PM, Ben Johnson wrote:
> I'm working to configure automated quota notifications in Dovecot and am
> wondering if it is possible to send a warning message to a user when
> message delivery fails because the user is over-quota.
> 
> I already have the following directives configured:
> 
>   quota_warning = storage=95%% quota-warning 95 %u %d
>   quota_warning2 = storage=80%% quota-warning 80 %u %d
>   quota_warning3 = -storage=100%% quota-below below %u %d
> 
> These seem to function as expected, but the problem I'm facing is that
> when a message is large enough to take the user's quota from, say, 84%
> to over 100%, the message is rejected (as expected), but the user is
> never warned that a message failed to be delivered because it would have
> put him over 100% usage.
> 
> I suppose that I'm looking for a trigger that is tripped when delivering
> the message *would* put the user over-quota (as opposed to *does in
> fact* put the user over).
> 
> The sender receives an automated rejection message already. I'm
> wondering if the recipient can be warned at the same time. (I'm not
> concerned about further increasing the user's quota consumption with the
> warning message; I would set noenforcing to 1.)
> 
> Is this possible? Any help is much appreciated. Thanks in advance.
> 
> -Ben
> 

You can give some additional space = the max message size allowed in
your email system, like in the example:
http://wiki2.dovecot.org/Quota/Configuration#line-1-5
(change Trash to Inbox). Then the message would be accepted, and the
user would get the normal 100% notice.

Ken

-- 
Ken Anderson
Pacific Internet - http://www.pacific.net

Re: [Dovecot] v2.2.3 released

[Ken A]

On 6/18/2013 10:54 AM, Timo Sirainen wrote:
> On 18.6.2013, at 18.22, Ken A  wrote:
> 
>>> sdbox/mdbox users: Don't upgrade just yet.. It seems it may
>>> cause "Extension header update points outside header size" errors
>>> that don't fix themselves. (Bug 1: Causing this error in the
>>> first place, bug 2: not being able to fix it automatically.)
>> 
>> Hi Timo, The latest from http://hg.dovecot.org/dovecot-2.2/ seems
>> to fix the dsync errors I was seeing with 2.2.2. Mostly "Error:
>> Mailbox INBOX sync: mailbox_delete failed: INBOX can't be
>> deleted."
> 
> Normally that shouldn't happen in the first place! But yeah, that fix
> was done intentionally. But if INBOX is being deleted all the time
> with you there's something wrong.

I suspect I'm causing breakage of metadata. I'm preparing to migrate to
mdbox from mbox, so I'm rsyncing mboxes to a new server and then running
dsync -R -u  backup mbox:/
The INBOXes that were generating this error were those that I'd opened
using an IMAP client on the new server (testing mailboxes) between
rsync/dsync runs.
Thanks,
Ken

> 
>> Is the extension header bug fixed with yesterday's patch: 
>> http://hg.dovecot.org/dovecot-2.2/rev/3056feb418b1 ?
> 
> Today's patch :) But yes, that one. And since it happens only with
> corrupted dboxes anyway I guess it's not actually that bad. Oh,
> almost forgot to fix this for sdbox also:
> http://hg.dovecot.org/dovecot-2.2/rev/07642120b6ea
> 

-- 
Ken Anderson
Pacific Internet - http://www.pacific.net

Re: [Dovecot] v2.2.3 released

[Ken A]

On 6/17/2013 8:06 AM, Timo Sirainen wrote:
> On 17.6.2013, at 0.41, Timo Sirainen  wrote:
> 
>> http://dovecot.org/releases/2.2/dovecot-2.2.3.tar.gz 
>> http://dovecot.org/releases/2.2/dovecot-2.2.3.tar.gz.sig
>> 
>> This is a pretty important upgrade for v2.2 users, because of the
>> IMAP ENVELOPE reply fix.
> 
> sdbox/mdbox users: Don't upgrade just yet.. It seems it may cause
> "Extension header update points outside header size" errors that
> don't fix themselves. (Bug 1: Causing this error in the first place,
> bug 2: not being able to fix it automatically.)

Hi Timo,
The latest from http://hg.dovecot.org/dovecot-2.2/ seems to fix the
dsync errors I was seeing with 2.2.2. Mostly "Error: Mailbox INBOX sync:
mailbox_delete failed: INBOX can't be deleted."

Is the extension header bug fixed with yesterday's patch:
http://hg.dovecot.org/dovecot-2.2/rev/3056feb418b1 ?

Thanks,
Ken Anderson

> If you're already getting those errors, attached a workaround patch.
> Probably happens only to POP3 users. I'm not sure yet how to
> reproduce this.
> 
> 
> 
> 
> 

-- 
Ken Anderson
Pacific Internet - http://www.pacific.net

Re: [Dovecot] userdb section order seems important in config file

[Ken A]

Ah, it looks like this is by design. Great.
http://wiki2.dovecot.org/Authentication/MultipleDatabases

It wasn't mentioned as a solution on
http://wiki2.dovecot.org/Quota/Configuration#passwd-file so I missed it.

Ken

On 5/24/2013 2:09 PM, Ken A wrote:
> Hi Timo, et al,
> 
> I'm using Dovecot 2.2.2, with dict file based quota and a quota_rule in
> dovecot.conf. I have userdb configured like so:
> 
> userdb {
>   driver = passwd-file
>   args = /etc/dovecot/passwd-file
> }
> userdb {
>   driver = passwd
> }
> 
> I get the behavior I want with a single line in the passwd-file for any
> users that I'd like to override the default quota. BUT, if I list the
> userdb sections in the reverse order, with the driver=passwd first, then
> the quotas in passwd-file are ignored.
> 
> I have a feeling I am counting on undocumented behavior that might
> change in the future. Or is this okay to depend on?
> 
> Thanks,
> Ken
> 

-- 
Ken Anderson
Pacific Internet - http://www.pacific.net

[Dovecot] userdb section order seems important in config file

[Ken A]

Hi Timo, et al,

I'm using Dovecot 2.2.2, with dict file based quota and a quota_rule in
dovecot.conf. I have userdb configured like so:

userdb {
  driver = passwd-file
  args = /etc/dovecot/passwd-file
}
userdb {
  driver = passwd
}

I get the behavior I want with a single line in the passwd-file for any
users that I'd like to override the default quota. BUT, if I list the
userdb sections in the reverse order, with the driver=passwd first, then
the quotas in passwd-file are ignored.

I have a feeling I am counting on undocumented behavior that might
change in the future. Or is this okay to depend on?

Thanks,
Ken

-- 
Ken Anderson
Pacific Internet - http://www.pacific.net

Re: [Dovecot] repeating dsync - questions

[Ken A]

Maybe take a look at "vmstat 2" and see if i/o is blocking a lot or you
are hitting swap space? Someone else may be more informed about the
inner workings of dsync and how it handles maildir as opposed to mbox.
Best of luck,
Ken


On 4/6/2013 1:09 AM, Bob Gustafson wrote:
> Something must be wrong with my setup. It took 14+ hours for the first
> dsync pass and it hasn't finished yet on the rerun to pick up the latest
> mail (around 12+ hours). I have about 9 GB of mail! The destination
> system is not fast, but..
> 
> Maybe I will wait for the 2.2 release..
> 
> Thanks for your response.
> 
> Bob G
> 
> On Fri, 2013-04-05 at 22:45 -0500, Ken A wrote:
>> It's about 300GB of mboxes (I don't know how many messages) and dsync
>> took about 3 hours from scratch in the destination. But, when I sync
>> more frequently, it's much quicker, and depends on the number of
>> changes. With no changes it runs in about 15 min.
>> Ken A.
>>
>>
>> On 4/5/2013 1:54 PM, Bob Gustafson wrote:
>>> I tried that a week or so ago, with a 'dsync -R backup', but got the
>>> funny named directories, so I read more and am trying the 'dsync
>>> mirror' which doesn't require the -R.
>>>
>>> How long does the sync step take for you? (Normalize to # of
>>> messages..)
>>>
>>> Bob G
>>>
>>> On Apr 5, 2013, at 13:48, Ken A  wrote:
>>>
>>>> I've found that when going in one direction, using "backup -R"
>>>> rather than mirror, works better. I'm going from mbox to sdbox, but
>>>> doing roughly the same thing you are, rsync and then dsync.
>>>>
>>>> Ken A.
>>>>
>>>> On 4/5/2013 10:40 AM, Bob Gustafson wrote:
>>>>>
>>>>> I'm moving/converting email from one system to another
>>>>>
>>>>> The source system is: Dual core x86_64 6GB memory 180 GB raid1
>>>>> disks ext4 Fedora 9 Dovecot 1.0.15 Maildir format
>>>>>
>>>>> The destination system is: Dual core x86_64 2GB memory 1TB raid1
>>>>> disks ext4 Fedora 18 Dovecot 2.1.15 sdbox format
>>>>>
>>>>> I am moving mail in a series of steps:
>>>>>
>>>>> 0) cd ( to mail user home directory ) mkdir Maildir
>>>>>
>>>>> 1) # rsync mail from source system (hoho4) to current (hoho0) 
>>>>> time rsync -arv --times hoho4:/home/bobgus/Maildir/
>>>>> /home/bobgus/Maildir/
>>>>>
>>>>> This step takes about 37 minutes the first time
>>>>>
>>>>> 2) # dsync mirror/convert to sdbox format on current system time
>>>>> dsync mirror maildir:~/Maildir
>>>>>
>>>>> This step takes about 858 minutes (!!)
>>>>>
>>>>> Looking at -D messages indicates that dsync is deciding between 
>>>>> duplicates much of the time.
>>>>>
>>>>> Looking at the results in a mail browser (Evolution), it seems
>>>>> fine, although the latest mails are not there.
>>>>>
>>>>> 3) # pick up more current mail - use same script as in 1) time
>>>>> rsync -arv --times hoho4:/home/bobgus/Maildir/
>>>>> /home/bobgus/Maildir/
>>>>>
>>>>> This step takes about 5 minutes although this varies depending on
>>>>> the amount of new mail.
>>>>>
>>>>> 4) # repeat the 2) dsync step time dsync mirror
>>>>> maildir:~/Maildir
>>>>>
>>>>> This is where things look peculiar. When I look at the directory
>>>>> of sdbox/mailboxes, I see duplicated directories
>>>>>
>>>>> [bobgus@hoho0 sdbox]$ cd mailboxes [bobgus@hoho0 mailboxes]$ ls 
>>>>> Apple Mail To Do Apple Mail To
>>>>> Do_2a47983780615e517960ba55d82c Deleted Messages Deleted
>>>>> Messages_2847983780615e517960ba55d82c Drafts 
>>>>> Drafts_2447983780615e517960ba55d82c Important 
>>>>> Important_2947983780615e517960ba55d82c INBOX 
>>>>> INBOX_1547983780615e517960ba55d82c Sent 
>>>>> Sent_2547983780615e517960ba55d82c Trash 
>>>>> Trash_2747983780615e517960ba55d82c
>>>>>
>>>>> The 2nd dsync step has not completed yet. I'm wondering whether
>>>>> the extra directory will be magically moved into the older
>>>>> directory.
>>>>>
>>>>> (This is unlikely because the file names are duplicated in the
>>>>> new directory)
>>>>>
>>>>> ---
>>>>>
>>>>> Is this expected behavior? Are there command changes I can make
>>>>> to speed up the process? Eliminate the duplicate directories?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>> -- Ken Anderson Pacific Internet - http://www.pacific.net
>>>
>>
> 
> 

-- 
Ken Anderson
Pacific Internet - http://www.pacific.net

Re: [Dovecot] repeating dsync - questions

[Ken A]

It's about 300GB of mboxes (I don't know how many messages) and dsync
took about 3 hours from scratch in the destination. But, when I sync
more frequently, it's much quicker, and depends on the number of
changes. With no changes it runs in about 15 min.
Ken A.


On 4/5/2013 1:54 PM, Bob Gustafson wrote:
> I tried that a week or so ago, with a 'dsync -R backup', but got the
> funny named directories, so I read more and am trying the 'dsync
> mirror' which doesn't require the -R.
> 
> How long does the sync step take for you? (Normalize to # of
> messages..)
> 
> Bob G
> 
> On Apr 5, 2013, at 13:48, Ken A  wrote:
> 
>> I've found that when going in one direction, using "backup -R"
>> rather than mirror, works better. I'm going from mbox to sdbox, but
>> doing roughly the same thing you are, rsync and then dsync.
>> 
>> Ken A.
>> 
>> On 4/5/2013 10:40 AM, Bob Gustafson wrote:
>>> 
>>> I'm moving/converting email from one system to another
>>> 
>>> The source system is: Dual core x86_64 6GB memory 180 GB raid1
>>> disks ext4 Fedora 9 Dovecot 1.0.15 Maildir format
>>> 
>>> The destination system is: Dual core x86_64 2GB memory 1TB raid1
>>> disks ext4 Fedora 18 Dovecot 2.1.15 sdbox format
>>> 
>>> I am moving mail in a series of steps:
>>> 
>>> 0) cd ( to mail user home directory ) mkdir Maildir
>>> 
>>> 1) # rsync mail from source system (hoho4) to current (hoho0) 
>>> time rsync -arv --times hoho4:/home/bobgus/Maildir/
>>> /home/bobgus/Maildir/
>>> 
>>> This step takes about 37 minutes the first time
>>> 
>>> 2) # dsync mirror/convert to sdbox format on current system time
>>> dsync mirror maildir:~/Maildir
>>> 
>>> This step takes about 858 minutes (!!)
>>> 
>>> Looking at -D messages indicates that dsync is deciding between 
>>> duplicates much of the time.
>>> 
>>> Looking at the results in a mail browser (Evolution), it seems
>>> fine, although the latest mails are not there.
>>> 
>>> 3) # pick up more current mail - use same script as in 1) time
>>> rsync -arv --times hoho4:/home/bobgus/Maildir/
>>> /home/bobgus/Maildir/
>>> 
>>> This step takes about 5 minutes although this varies depending on
>>> the amount of new mail.
>>> 
>>> 4) # repeat the 2) dsync step time dsync mirror
>>> maildir:~/Maildir
>>> 
>>> This is where things look peculiar. When I look at the directory
>>> of sdbox/mailboxes, I see duplicated directories
>>> 
>>> [bobgus@hoho0 sdbox]$ cd mailboxes [bobgus@hoho0 mailboxes]$ ls 
>>> Apple Mail To Do Apple Mail To
>>> Do_2a47983780615e517960ba55d82c Deleted Messages Deleted
>>> Messages_2847983780615e517960ba55d82c Drafts 
>>> Drafts_2447983780615e517960ba55d82c Important 
>>> Important_2947983780615e517960ba55d82c INBOX 
>>> INBOX_1547983780615e517960ba55d82c Sent 
>>> Sent_2547983780615e517960ba55d82c Trash 
>>> Trash_2747983780615e517960ba55d82c
>>> 
>>> The 2nd dsync step has not completed yet. I'm wondering whether
>>> the extra directory will be magically moved into the older
>>> directory.
>>> 
>>> (This is unlikely because the file names are duplicated in the
>>> new directory)
>>> 
>>> ---
>>> 
>>> Is this expected behavior? Are there command changes I can make
>>> to speed up the process? Eliminate the duplicate directories?
>>> 
>>> 
>>> 
>>> 
>> 
>> -- Ken Anderson Pacific Internet - http://www.pacific.net
> 

-- 
Ken Anderson
Pacific Internet - http://www.pacific.net

Re: [Dovecot] repeating dsync - questions

[Ken A]

I've found that when going in one direction, using "backup -R" rather
than mirror, works better. I'm going from mbox to sdbox, but doing
roughly the same thing you are, rsync and then dsync.

Ken A.

On 4/5/2013 10:40 AM, Bob Gustafson wrote:
> 
> I'm moving/converting email from one system to another
> 
> The source system is:
> Dual core x86_64
> 6GB memory
> 180 GB raid1 disks ext4
> Fedora 9
> Dovecot 1.0.15
> Maildir format
> 
> The destination system is:
> Dual core x86_64
> 2GB memory
> 1TB raid1 disks ext4
> Fedora 18
> Dovecot 2.1.15
> sdbox format
> 
> I am moving mail in a series of steps:
> 
> 0) cd ( to mail user home directory )
>mkdir Maildir
> 
> 1) # rsync mail from source system (hoho4) to current (hoho0)
>time rsync -arv --times hoho4:/home/bobgus/Maildir/ /home/bobgus/Maildir/
> 
> This step takes about 37 minutes the first time
> 
> 2) # dsync mirror/convert to sdbox format on current system
>time dsync mirror maildir:~/Maildir
> 
> This step takes about 858 minutes (!!)
> 
> Looking at -D messages indicates that dsync is deciding between
> duplicates much of the time.
> 
> Looking at the results in a mail browser (Evolution), it seems fine,
> although the latest mails are not there.
> 
> 3) # pick up more current mail - use same script as in 1)
>time rsync -arv --times hoho4:/home/bobgus/Maildir/ /home/bobgus/Maildir/
> 
> This step takes about 5 minutes although this varies depending on the
> amount of new mail.
> 
> 4) # repeat the 2) dsync step
>time dsync mirror maildir:~/Maildir
> 
> This is where things look peculiar. When I look at the directory of
> sdbox/mailboxes, I see duplicated directories
> 
> [bobgus@hoho0 sdbox]$ cd mailboxes
> [bobgus@hoho0 mailboxes]$ ls
> Apple Mail To Do
> Apple Mail To Do_2a47983780615e517960ba55d82c
> Deleted Messages
> Deleted Messages_2847983780615e517960ba55d82c
> Drafts
> Drafts_2447983780615e517960ba55d82c
> Important
> Important_2947983780615e517960ba55d82c
> INBOX
> INBOX_1547983780615e517960ba55d82c
> Sent
> Sent_2547983780615e517960ba55d82c
> Trash
> Trash_2747983780615e517960ba55d82c
> 
> The 2nd dsync step has not completed yet. I'm wondering whether the
> extra directory will be magically moved into the older directory.
> 
> (This is unlikely because the file names are duplicated in the new
> directory)
> 
> ---
> 
> Is this expected behavior? Are there command changes I can make to speed
> up the process? Eliminate the duplicate directories?
> 
> 
> 
> 

-- 
Ken Anderson
Pacific Internet - http://www.pacific.net

Re: [Dovecot] dsync mirror to convert from mbox (1.2x) to mdbox (2.2)

[Ken A]

On 3/31/2013 3:20 AM, Timo Sirainen wrote:
> On 22.3.2013, at 0.31, Ken A  wrote:
> 
>> I'm converting mail from mbox to mdbox. I've copied mail over to a
>> new system, and now I'm running dsync like this:
>> 
>> dsync -u username mirror 
>> mbox:/folder_location/username:INBOX=/inbox_location/username
>> 
>> That seemed to work, with a couple minor errors:
>> 
>>> dsync(username): Warning: Our dotlock file
>>> /mail_location/username.lock was deleted (locked 20 secs ago,
>>> touched 20 secs ago) dsync(username): Error:
>>> file_dotlock_delete() failed with mbox file
>>> /mail_location/username: No such file or directory
>> 
>> Do I also need to specify where dovecot's indexes are for the old
>> mbox files, or is this not needed for mbox conversions (when they
>> are the source)?
> 
> 
> Indexes aren't required, but looks like it takes a while to run dsync
> and you have another process that decides 20 seconds is long enough
> for a timeout and deletes the lock. Indexes would make it somewhat
> faster. Do you have mbox_dotlock_change_timeout=20? That's too low.
> 

I installed RC2.3, and am now using "dsync -R -u username backup mbox:"
and it seems to work fine now.

The only lock files would be those created by dsync, since this is all
just an rsync'd copy of mboxes that previously lived in /var/spool/mail
on another box. There are no mbox specific configurations in dovecot.

Thanks,
Ken

-- 
Ken Anderson
Pacific Internet - http://www.pacific.net

[Dovecot] dsync mirror to convert from mbox (1.2x) to mdbox (2.2)

[Ken A]

I'm converting mail from mbox to mdbox.
I've copied mail over to a new system, and now I'm running dsync like this:

dsync -u username mirror
mbox:/folder_location/username:INBOX=/inbox_location/username

That seemed to work, with a couple minor errors:

> dsync(username): Warning: Our dotlock file /mail_location/username.lock was 
> deleted (locked 20 secs ago, touched 20 secs ago)
> dsync(username): Error: file_dotlock_delete() failed with mbox file 
> /mail_location/username: No such file or directory

Do I also need to specify where dovecot's indexes are for the old mbox
files, or is this not needed for mbox conversions (when they are the
source)?

Thanks,
Ken A.

[Dovecot] 1.x with mbox files to 2.x with mdbox, I think?

[Ken A]

Hi, I have a couple questions about migrating from 1.x to 2.x

I'm needing to support larger mailbox sizes and mdbox seems like the
right format to use to get larger size boxes and maintain, or possibly
even increase speed. Is that correct?

Am I correct that the best way to accomplish this migration is to copy
the mboxes over to the new server and then use dsync to convert them to
mdbox?

Thanks for your thoughts!
Ken


-- 
Ken Anderson
Pacific Internet - http://www.pacific.net
Latest Pacific.Net Status - http://twitter.com/pacnetstatus

Re: [Dovecot] Dovecot Virtual Users with Sendmail

[Ken A]

Dovecot has no problem with virtual users, but Sendmail's idea of
virtual users is not quite as 'virtual'.

One possible solution: http://www.cs.fsu.edu/~langley/USVH-tr.pdf
That was 2007.. so perhaps somebody has written this up in a more
concise way..

or you can wrestle with ldap. :-(

or switch MTAs.

Ken A.

On 12/30/2012 9:19 PM, dormitionsk...@hotmail.com wrote:
> I'm trying to get Dovecot working with Virtual Users and Sendmail on 
> OpenIndiana 151a7.  OpenIndiana is the Open Source continuation of 
> OpenSolaris after Oracle tried to kill it off a couple of years ago shortly 
> after acquiring Sun Microsystems.
> 
> I'm not sure if this is so much a Dovecot problem, as much of a Sendmail 
> problem.  I thought this would probably be a better list to start with, 
> though.  If not, please let me know.
> 
> I initially used this Dovecot wiki page to approach this:
> Simple Virtual User Installation
> 
> http://wiki2.dovecot.org/HowTo/SimpleVirtualInstall
> 
> I had to make few minor changes, but I don't think anything significant to 
> the problem I'm having.
> 
> I can log into SquirrelMail with both virtual users, and with system users.
> 
> I can send mail to system users from both virtual and system users.
> 
> I can see the mail that I sent in the Sent Messages folder for both types of 
> users.
> 
> I just can't send mail to virtual users.  When I try to, I get the following 
> message:
> 
> ERROR:
> Message not sent. Server replied:
> Requested action not taken: mailbox unavailable
> 550 5.1.1 ... User unknown
> My configuration info is as follows:
> 
> systemuser@openindiana:~# /usr/local/sbin/dovecot --version
> 2.1.12
> 
> systemuser@openindiana:~# /usr/local/sbin/dovecot -n
> # 2.1.12: /usr/local/etc/dovecot/dovecot.conf
> # OS: SunOS 5.11 i86pc  
> auth_verbose = yes
> base_dir = /var/run/dovecot/
> disable_plaintext_auth = no
> first_valid_uid = 101
> info_log_path = /var/log/dovecot-info.log
> log_path = /var/log/dovecot.log
> mail_location = mbox:/home/vmail/%u/mail:INBOX=/var/mail/%u
> passdb {
>   args = /usr/local/etc/dovecot/passwd
>   driver = passwd-file
> }
> pop3_uidl_format = %g
> protocols = imap pop3
> service auth {
>   unix_listener auth-userdb {
> group = vmail
> mode = 0600
> user = vmail
>   }
> }
> ssl = no
> userdb {
>   args = uid=vmail gid=vmail home=/home/vmail/%u
>   driver = static
> }
> protocol lda {
>   postmaster_address = systemuser@localhost
> }
> 
> ---
> 
> My dovecot password file looks something like this, though changed for 
> obvious security reasons:
> 
> systemuser:{PLAIN} systemuserPASS
> virtualuser1:{PLAIN}virtualuser1PASS
> virtualuser2:{PLAIN}virtualuser2PASS
> 
> 
> ---
> 
> I read on this page that I needed to put the following in my sendmail 
> dovecot.m4 file:
> 
> Dovecot LDA with Sendmail
> http://wiki2.dovecot.org/LDA/Sendmail
> 
> So this is my dovecot.m4 file, which shows up in my sendmail.cf file after 
> recompiling my sendmail.mc file.
> 
> ##*##
> ###   DOVECOT Mailer specification  ###
> ##*##
> Mdovecot,   P=/usr/local/libexec/dovecot/dovecot-lda, F=DFMPhnu9,
>  S=EnvFromSMTP/HdrFromSMTP, R=EnvToSMTP/HdrFromSMTP,
>  T=DNS/RFC822/X-Unix,
>  A=/usr/local/libexec/dovecot/dovecot-lda -d $u
>  
> 
> ---
> 
> Finally, this is my sendmail.mc file:
> 
> divert(-1)
> #
> # Copyright (c) 1983 Eric P. Allman
> # Copyright (c) 1988, 1993
> # The Regents of the University of California.  All rights reserved.
> #
> # Copyright 2009 Sun Microsystems, Inc.  All rights reserved.
> # Use is subject to license terms.
> #
> #  This is a configuration file for SunOS 5.8 (a.k.a. Solaris 8) and later
> #  subsidiary machines.  It has support for local and SMTP mail.  The
> #  confFALLBACK_SMARTHOST macro is enabled, which means that messages will
> #  be sent to that host (which is set to mailhost.$m [$m is the local domain])
> #  if MX records are unavailable.  A short-cut rule is also defined, which
> #  says if the recipient host is in the local domain, send to it directly
> #  instead of the smart host.
> #
> #  If you want to customize this further, copy it to a name appropriate
> #  for your environment and do the modifications there.
> #
> 
> divert(0)dnl
> VERSIONID(`sendmail.mc (Sun)')
> OSTYPE(`solaris8')dnl
> DOMAIN(`solaris-generic')dnl
> define(`confFALLBACK_SMARTHOST', `mailhost$?m.$m$.')dnl
> 
> define(`PROCMAI

Re: [Dovecot] migration to dovecot pop3

[Ken A]

On 12/19/2012 6:59 PM, Joseph Tam wrote:
> Rajesh M writes:
> 
>> i am planning to migrate from qmail pop3 to dovecot pop3
>>
>> the number of users on the server is large over 5000
>> ...
>> also data volume is large - over 3.5 tb -- ie many users several 1000s of
>> emails in their inbox
> 
> If you stay on the list, you might find some people here whose
> mail metrics are several orders of magnitude larger than this.
> 
>> 1) will the migration cause all the emails on the server to re-download
>> ...
>> 4) has anybody migrated from qmail pop3 to dovecot pop3 successfully
>> without emails being downloaded again
> 
> A proper migration will avoid this.  Google is your friend
> 
> http://lmgtfy.com/q=POP+qmail+to+dovecot+migration
> 
> The top hits provide exactly what you're looking for.
> 
>> 2) can dovecot be configured to allow only one download at a time - pop3
>> locking with time out setting that can be customized

We migrated 5000+ from qpopper years ago. Dovecot is sooo much better on
system load. Yes, locking and timeouts are in the config file.

>>
>> 3) can dovecot pop3 be configured so that it will allow download of
>> emails
>> only of the last say 15 days ie even if the same pop3 user is
>> configure on
>> another machine it will download only messages of the last 15 days

That's not part of pop3, so no, it's not possible with dovecot alone.
You could probably work out a plugin to move read mail after x days, or
there may be one out there ? Or some imap robot could do it..
Ken A.
Pacific.Net


> I can't answer this, but if this is an attempt to prevent mass download,
> the answer to 1+4 makes these questions irrelevant.
> 
> Joseph Tam 
> 

-- 
Ken Anderson
Pacific Internet - http://www.pacific.net
Latest Pacific.Net Status - http://twitter.com/pacnetstatus

Re: [Dovecot] Problem managing mbox

[Ken A]

I have seen this error msg too. Deleting the index files 'fixes' the 
error messages in the log. The indexes seem to get corrupted sometimes, 
when the client is logged doing: copy dest=Trash, delete, expunge


Ken


On 3/19/2012 9:00 AM, Timo Sirainen wrote:

On Sat, 2012-03-17 at 20:42 +0100, PSTM wrote:

Hello,

I have a problem with dovecot. seems that do not erase mail that mail
client request to be erased.


Are you sure the clients have actually issued the EXPUNGE command,
rather than simply marked the mail with \Deleted flag?


And I have this errors:

  Error: Next message unexpectedly corrupted in mbox file

Info:


mbox code isn't perfect, but if this doesn't happen often it shouldn't
matter much. doveconf -n output might have been helpful in giving more
suggestions.





--
Ken Anderson
Pacific Internet - http://www.pacific.net
Latest Pacific.Net Status - http://twitter.com/pacnetstatus

[Dovecot] mdbox and pop3 locking

[Ken A]

With mdbox, what does dovecot lock when "pop3_lock_session(pop3): yes"?

Specifically, I'm wondering if Dovecot LDA is able to deliver mail when 
a session is locked, if using mdbox, or if it will tempfail until the 
session is unlocked?


Thanks,
Ken

--
Ken Anderson
Pacific Internet - http://www.pacific.net

Re: [Dovecot] Attacking Dovecot

[Ken A]

That's all normal activity (failed logins) for any internet facing 
machine. They may be dictionary attacks, or not... If they get on your 
nerves, block them. Strong passwords will help more.


Also, it's likely that you have forged mail coming in from outside, and 
not really "spam from local users" ?


If it is really locally generated, then disable the account.

Ken


On 9/9/2011 4:45 PM, Nikos Papadopoulos wrote:

Hello,



I am using Dovecot ver.1.0.7 on an x86 server with RedHat Linux Enterprise 5
and the following configuration:



# 1.0.7: /etc/dovecot.conf

protocols: pop3

login_dir: /var/run/dovecot/login

login_executable: /usr/libexec/dovecot/pop3-login

mail_location: mbox:~/mail:INBOX=/var/mail/%u

mail_executable: /usr/libexec/dovecot/pop3

mail_plugin_dir: /usr/lib/dovecot/pop3

pop3_client_workarounds: outlook-no-nuls oe-ns-eoh

auth default:

   passdb:

 driver: pam

   userdb:

 driver: passwd





It seems that my mail server is being attacked by someone who tries to
retrieve users' credentials. Please read below an output of logwatch.



dovecot-auth: pam_succeed_if(dovecot:auth): error retrieving information
about

user sandra

  dovecot-auth: pam_succeed_if(dovecot:auth): error retrieving information
about

user tanya

  dovecot-auth: pam_succeed_if(dovecot:auth): error retrieving information
about

user tanya

  dovecot-auth: pam_succeed_if(dovecot:auth): error retrieving information
about

user dark

  dovecot-auth: pam_succeed_if(dovecot:auth): error retrieving information
about

user dark

  dovecot-auth: pam_succeed_if(dovecot:auth): error retrieving information
about

user gibson

  dovecot-auth: pam_succeed_if(dovecot:auth): error retrieving information
about

user frank

  dovecot-auth: pam_succeed_if(dovecot:auth): error retrieving information
about

user frank







Besides, some of the local users receive "spam" emails, which seem to be
sent by another local user.



Please assist me on how to prevent the aforementioned attack.



Best Regards,



Nikos






--
Ken Anderson
Pacific Internet - http://www.pacific.net
Latest Pacific.Net Status - http://twitter.com/pacnetstatus

Re: [Dovecot] unsubscribe

[Ken A]

Please consider brain cells before sending it too..

On 3/4/2011 5:18 AM, Rob Coward wrote:




Please consider the environment before printing this email.

Game Stores Group Limited, registered number 1937170, registered in England



--
Ken Anderson
Pacific Internet - http://www.pacific.net

Re: [Dovecot] POP3 Slow

[Ken A]

On 10/8/2010 11:04 PM, Avinash Mishra wrote:

Hi

I have tried out the Performance Tuning measures, I do not think that an

upgrade is going to help as most of the features can be incorporated

with the current version itself. ( already done that btw )



Well, to keep us from wasting our time, and to get relevant answers 
efficiently, it's good practice to outline your hardware, and what you 
have already tried in your first post. What does your hardware 
utilization look like?


Ken


I am unable to find any anomalies in the trace, or am I missing something ?

The only part that seems to be not ok, is the EAGAIN -1 return from the write() 
function.

I did run a protocol analyzer trace on the TCP transaction, when the POP3

clients were connecting and discovered a lot of Re-Transmissions and Duplicate

ACK packets.


--- On Fri, 8/10/10, Ken A  wrote:

From: Ken A
Subject: Re: [Dovecot] POP3 Slow
To: dovecot@dovecot.org
Date: Friday, 8 October, 2010, 7:24 PM

Have you tried the suggestions on improving performance in the wiki?
http://wiki.dovecot.org/PerformanceTuning
You will need to upgrade to take advantage of several of them..

Ken


On 10/7/2010 11:40 PM, Avinash Mishra wrote:

Hi,



I am running both IMAP/POP3 services. Recently, the POP3 users started 
complaining of slow performance.



I tried to resolve  it by advising them to trim their inbox, but to no avail.



Kindly advise as I am not able to resolve this issue.



dovecot -n:



# 1.0.7: /etc/dovecot.conf

ssl_cert_file: /etc/dovecot.cert.pem

ssl_key_file: /etc/dovecot.key.pem

login_dir: /var/run/dovecot/login

login_executable(default): /usr/libexec/dovecot/imap-login

login_executable(imap): /usr/libexec/dovecot/imap-login

login_executable(pop3): /usr/libexec/dovecot/pop3-login

mail_location: maildir:~/Maildir

mail_executable(default): /usr/libexec/dovecot/imap

mail_executable(imap): /usr/libexec/dovecot/imap

mail_executable(pop3): /usr/libexec/dovecot/pop3

mail_plugin_dir(default): /usr/lib/dovecot/imap

mail_plugin_dir(imap): /usr/lib/dovecot/imap

mail_plugin_dir(pop3): /usr/lib/dovecot/pop3

pop3_client_workarounds(default):

pop3_client_workarounds(imap):

pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh

auth default:

 mechanisms: plain login

 passdb:

   driver: pam

 userdb:

   driver: passwd



=



sstrace -tt -o log -p:



18:45:55.425819 gettimeofday({1286457355, 425856}, {4294966966, 0}) = 0

18:45:55.425901 write(1, "hQIKACgAoAKACgAoAKACgAoASgBaBhQI"..., 1683) = 1683

18:45:55.425987 setsockopt(1, SOL_TCP, TCP_CORK, [1], 4) = 0

18:45:55.426044 pread64(9, "UAFABQAUAFABQBR+zxT31x5qBsBcUAOu"..., 4096, 20480) 
= 4096

18:45:55.426137 write(1, "\r\nAUAFABQAUAFABQAUAFABQAUAFABQAU"..., 4134) = 4134

18:45:55.426204 pread64(9, "/AJ5p+VAg+zwd4k/7\n5FAB9nh/55J/3y"..., 4096, 24576) 
= 4096

18:45:55.426298 write(1, "\r\nAC+UmP4v++jR8gsHkp7/APfRosuwWI"..., 4134) = 4134

18:45:55.426373 pread64(9, "FA\nBQAUAFABQAUAFABQAUAFABQAUAFAB"..., 4096, 28672) 
= 4096

18:45:55.426466 write(1, "\r\nKBBQAUAFABQAUAFABQAUAFABQAUAFA"..., 4134) = 1729

18:45:55.426528 write(1, "rSs+4B+8/2f1oswD956r+VGow/eeq/lR"..., 2405) = -1 
EAGAIN (Resource temporarily unavailable)

18:45:55.426600 setsockopt(1, SOL_TCP, TCP_CORK, [0], 4) = 0

18:45:55.426674 gettimeofday({1286457355, 426704}, NULL) = 0

18:45:55.426752
poll([{fd=4, events=POLLIN|POLLPRI|POLLERR|POLLHUP|POLLNVAL}, {fd=2,
events=POLLERR|POLLHUP|POLLNVAL}, {fd=0,
events=POLLIN|POLLPRI|POLLERR|POLLHUP|POLLNVAL}, {fd=1,
events=POLLOUT|POLLERR|POLLHUP|POLLNVAL}], 4, 4839) = 1 ([{fd=1,
revents=POLLOUT}])

18:45:55.803792 gettimeofday({1286457355, 803819}, {4294966966, 0}) = 0

18:45:55.803867 write(1, "rSs+4B+8/2f1oswD956r+VGow/eeq/lR"..., 2405) = 2405

18:45:55.803943 setsockopt(1, SOL_TCP, TCP_CORK, [1], 4) = 0

18:45:55.803998 pread64(9, "BQAUAFABQAUAFABQAUAFABQAUAFABQAU"..., 4096, 32768) 
= 4096

18:45:55.804091 write(1, "\r\nAFABQAUAFABQAUAFABQAUAFABQAUAF"..., 4134) = 4134

18:45:55.804158 pread64(9, "QH+Dbzj0zQBbVCrjDfIFxtx+tAD6ACgA"..., 4096, 36864) 
= 4096

18:45:55.804251 write(1, "\r\njQBQgtryBSqPDknJJBJJ96ALe6QXCp"..., 4134) = 2221

18:45:55.804327 write(1, "86U5IzwPagCxcQtI0BXHyOGP0waAJ6AF"..., 1913) = -1 
EAGAIN (Resource temporarily unavailable)

18:45:55.804399 setsockopt(1, SOL_TCP, TCP_CORK, [0], 4) = 0

18:45:55.804449 gettimeofday({1286457355, 804466}, NULL) = 0

18:45:55.804497
poll([{fd=4, events=POLLIN|POLLPRI|POLLERR|POLLHUP|POLLNVAL}, {fd=2,
events=POLLERR|POLLHUP|POLLNVAL}, {fd=0,
events=POLLIN|POLLPRI|POLLERR|POLLHUP|POLLNVAL}, {fd=1,
events=POLLOUT|POLLERR|POLLHUP|POLLNVAL}], 4, 4461) = 1 ([{fd=1,
revents=POLLOUT}])

18:45:56.189834 gettimeofday({1286457356, 189861}, {4294966966, 0}) = 0

18:45:56.1

Re: [Dovecot] POP3 Slow

[Ken A]

Have you tried the suggestions on improving performance in the wiki?
http://wiki.dovecot.org/PerformanceTuning
You will need to upgrade to take advantage of several of them..

Ken


On 10/7/2010 11:40 PM, Avinash Mishra wrote:

Hi,



I am running both IMAP/POP3 services. Recently, the POP3 users started 
complaining of slow performance.



I tried to resolve  it by advising them to trim their inbox, but to no avail.



Kindly advise as I am not able to resolve this issue.



dovecot -n:



# 1.0.7: /etc/dovecot.conf

ssl_cert_file: /etc/dovecot.cert.pem

ssl_key_file: /etc/dovecot.key.pem

login_dir: /var/run/dovecot/login

login_executable(default): /usr/libexec/dovecot/imap-login

login_executable(imap): /usr/libexec/dovecot/imap-login

login_executable(pop3): /usr/libexec/dovecot/pop3-login

mail_location: maildir:~/Maildir

mail_executable(default): /usr/libexec/dovecot/imap

mail_executable(imap): /usr/libexec/dovecot/imap

mail_executable(pop3): /usr/libexec/dovecot/pop3

mail_plugin_dir(default): /usr/lib/dovecot/imap

mail_plugin_dir(imap): /usr/lib/dovecot/imap

mail_plugin_dir(pop3): /usr/lib/dovecot/pop3

pop3_client_workarounds(default):

pop3_client_workarounds(imap):

pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh

auth default:

   mechanisms: plain login

   passdb:

 driver: pam

   userdb:

 driver: passwd



=



sstrace -tt -o log -p:



18:45:55.425819 gettimeofday({1286457355, 425856}, {4294966966, 0}) = 0

18:45:55.425901 write(1, "hQIKACgAoAKACgAoAKACgAoASgBaBhQI"..., 1683) = 1683

18:45:55.425987 setsockopt(1, SOL_TCP, TCP_CORK, [1], 4) = 0

18:45:55.426044 pread64(9, "UAFABQAUAFABQBR+zxT31x5qBsBcUAOu"..., 4096, 20480) 
= 4096

18:45:55.426137 write(1, "\r\nAUAFABQAUAFABQAUAFABQAUAFABQAU"..., 4134) = 4134

18:45:55.426204 pread64(9, "/AJ5p+VAg+zwd4k/7\n5FAB9nh/55J/3y"..., 4096, 24576) 
= 4096

18:45:55.426298 write(1, "\r\nAC+UmP4v++jR8gsHkp7/APfRosuwWI"..., 4134) = 4134

18:45:55.426373 pread64(9, "FA\nBQAUAFABQAUAFABQAUAFABQAUAFAB"..., 4096, 28672) 
= 4096

18:45:55.426466 write(1, "\r\nKBBQAUAFABQAUAFABQAUAFABQAUAFA"..., 4134) = 1729

18:45:55.426528 write(1, "rSs+4B+8/2f1oswD956r+VGow/eeq/lR"..., 2405) = -1 
EAGAIN (Resource temporarily unavailable)

18:45:55.426600 setsockopt(1, SOL_TCP, TCP_CORK, [0], 4) = 0

18:45:55.426674 gettimeofday({1286457355, 426704}, NULL) = 0

18:45:55.426752
poll([{fd=4, events=POLLIN|POLLPRI|POLLERR|POLLHUP|POLLNVAL}, {fd=2,
events=POLLERR|POLLHUP|POLLNVAL}, {fd=0,
events=POLLIN|POLLPRI|POLLERR|POLLHUP|POLLNVAL}, {fd=1,
events=POLLOUT|POLLERR|POLLHUP|POLLNVAL}], 4, 4839) = 1 ([{fd=1,
revents=POLLOUT}])

18:45:55.803792 gettimeofday({1286457355, 803819}, {4294966966, 0}) = 0

18:45:55.803867 write(1, "rSs+4B+8/2f1oswD956r+VGow/eeq/lR"..., 2405) = 2405

18:45:55.803943 setsockopt(1, SOL_TCP, TCP_CORK, [1], 4) = 0

18:45:55.803998 pread64(9, "BQAUAFABQAUAFABQAUAFABQAUAFABQAU"..., 4096, 32768) 
= 4096

18:45:55.804091 write(1, "\r\nAFABQAUAFABQAUAFABQAUAFABQAUAF"..., 4134) = 4134

18:45:55.804158 pread64(9, "QH+Dbzj0zQBbVCrjDfIFxtx+tAD6ACgA"..., 4096, 36864) 
= 4096

18:45:55.804251 write(1, "\r\njQBQgtryBSqPDknJJBJJ96ALe6QXCp"..., 4134) = 2221

18:45:55.804327 write(1, "86U5IzwPagCxcQtI0BXHyOGP0waAJ6AF"..., 1913) = -1 
EAGAIN (Resource temporarily unavailable)

18:45:55.804399 setsockopt(1, SOL_TCP, TCP_CORK, [0], 4) = 0

18:45:55.804449 gettimeofday({1286457355, 804466}, NULL) = 0

18:45:55.804497
poll([{fd=4, events=POLLIN|POLLPRI|POLLERR|POLLHUP|POLLNVAL}, {fd=2,
events=POLLERR|POLLHUP|POLLNVAL}, {fd=0,
events=POLLIN|POLLPRI|POLLERR|POLLHUP|POLLNVAL}, {fd=1,
events=POLLOUT|POLLERR|POLLHUP|POLLNVAL}], 4, 4461) = 1 ([{fd=1,
revents=POLLOUT}])

18:45:56.189834 gettimeofday({1286457356, 189861}, {4294966966, 0}) = 0

18:45:56.189908 write(1, "86U5IzwPagCxcQtI0BXHyOGP0waAJ6AF"..., 1913) = 1913

18:45:56.189982 setsockopt(1, SOL_TCP, TCP_CORK, [1], 4) = 0

18:45:56.190039 pread64(9, "uJ5Y9W/wC+jTC4CNf9r/vo0XC4uwep/w"..., 4096, 40960) 
= 4096

18:45:56.190136 write(1, "\r\nMN/e/Siwai4b+9+lKwBtb++fyosAbW"..., 4134) = 4134

18:45:56.190204 pread64(9, "0pAGBQAUALQAUAJQAtA\nBQAlAC0AFABQ"..., 4096, 45056) 
= 4096

18:45:56.190295 write(1, "\r\nB5Ef939aLLsGvcPs8WMbBRoGvcPs8X"..., 4134) = 1253

18:45:56.190370 write(1, "gAoAKACgAoAKACgAoAKACgAoAKACgAoA"..., 2881) = -1 
EAGAIN (Resource temporarily unavailable)

18:45:56.190442 setsockopt(1, SOL_TCP, TCP_CORK, [0], 4) = 0

18:45:56.190492 gettimeofday({1286457356, 190510}, NULL) = 0

18:45:56.190540
poll([{fd=4, events=POLLIN|POLLPRI|POLLERR|POLLHUP|POLLNVAL}, {fd=2,
events=POLLERR|POLLHUP|POLLNVAL}, {fd=0,
events=POLLIN|POLLPRI|POLLERR|POLLHUP|POLLNVAL}, {fd=1,
events=POLLOUT|POLLERR|POLLHUP|POLLNVAL}], 4, 4075) = 1 ([{fd=1,
revents=POLLOUT}])

18:45:56.394306 gettimeofday({1286457356, 394338}, {4294966966, 0}) = 0

18:45:56.394394 write(1, "gAoAKACgAoAKACgAoAKACgAoAKACgAoA"..., 2881) = 2881

18:45:56.394

Re: [Dovecot] Fwd: NFS question

[Ken A]

On 9/30/2010 7:28 AM, Charles Marcus wrote:

On 2010-09-30 7:38 AM, Cor Bosman wrote:

Charles wrote:

Timo, can you at least clarify this - my understanding is that the
problem here is not inherent to dovecot, but to NFS caching, and that
dovecot is no more or less prone to having problems than any other
imap/pop3 server?



That is correct. Except that many other imap/pop servers do not
implement a message index to increase performance. The problem lies
in the index, not in the messages themselves.


Right, I knew that, just wasn't clear...


You can turn off the index completely, and then you'll be just as
slow as any other imap server and wont ever see this problem.


But we're talking strictly POP3 here, so indexes aren't useful (and
therefor can be disabled completely if all you are using dovecot for is
as a POP3 server) anyway, right?


Just to clarify, we are talking strictly pop3 w/maildir. Indexing is 
quite useful for pop3 w/mbox.


Ken
Pacific.Net




As for IMAP, I think Timo also made it clear that if you weren't using
the dovecot LDA+index updating, or if the indexes weren't stored on NFS,
then IMAP didn't suffer the problem/issue as well... ?

I'm just glad I don't have to use NFS, so I'm asking just for the sake
of knowing (and in case I ever do need to use NFS)... :)



--
Ken Anderson
Pacific Internet - http://www.pacific.net

Re: [Dovecot] Dovecot pop3 segfault problems

[Ken A]

On 9/24/2010 5:27 AM, enid vx wrote:


and now, can you respond to Timos request for a backtrace:

On 2010-09-21 10:16 AM, Timo Sirainen  wrote:

Are these coming only from pop3 processes? Are you running imap? It's
anyway difficult to say anything about segfaults without a gdb
backtrace. See http://dovecot.org/bugreport.html how to get one.


--



Hi all,

after the update I noticed that the error messages didn't go away for a
little time. Also seeing the high load and high kernel cpu usage I did some
changes to the /etc/fstab (adding noatime,nodiratime) to /var /home
partitions.
Also the dovecot -n now is like this:
auth_mechanisms = plain login
default_login_user = dovecot
disable_plaintext_auth = no
dotlock_use_excl = yes
mail_access_groups = mail
mail_fsync = never
mail_location = mbox:~/.:INBOX=/var/mail/%u
mmap_disable = yes
passdb {
   driver = pam
}
ssl_cert =

You might try running 'vmstat 2', 'iostat -x', etc..
It sounds to me like a disk i/o problem, but we are just guessing unless 
we know more about the hardware and utilization.
A simple fix might be to add a fast local drive and move /var/mail to 
it's own spindle, assuming you don't have this configuration already.


Ken



Again thank you for your support.
Enid



--
Ken Anderson
Pacific Internet - http://www.pacific.net

Re: [Dovecot] STARTTLS does not seem to work

[Ken A]

On 5/24/2010 4:46 PM, Phil Howard wrote:

On Mon, May 24, 2010 at 17:31, Mike Abbott  wrote:

Well, that kinda complicates a "STARTTLS tunnel"


Perhaps you might be interested in these commands.  I'm not sure about their 
portability but they work tolerably well in scripts on Mac OS X 10.6.

$ openssl s_client -connect yourhost:imap -starttls imap
$ openssl s_client -connect yourhost:pop3 -starttls pop3

$ openssl s_client -connect yourhost:imaps
$ openssl s_client -connect yourhost:pop3s

$ openssl s_client -connect yourhost:smtp -starttls smtp



Yeah, that can be used, perhaps best with expect or pexpect.  I'm
hoping to find tools that can do basic email functions at a higher
level, where the user of the tool does not need to know the protocol
details, but only needs to consider the same kinds of configuration
aspects that configuring a regular email client involves (except
without all the misleading and often erroneous terminology used by
these GUI client developers ... such as "TLS" for STARTTLS/STLS on
clear ports, and "SSL" for wrapped/tunneled TLS/SSL connections on
always-encrypted ports as used in Evolution).


Mail::POP3Client works pretty well.
Net::IMAP::Simple looks easy too, but I've not used it.
Ken




At some point I think I need to learn the OpenSSL library API for C so
I can write some command line tool apps of my own with it (now we're
getting well off the Dovecot topic).



--
Ken Anderson
Pacific Internet - http://www.pacific.net

Re: [Dovecot] Thunderbird very slow startup, 1.2.11, mbox, postfix local delivery to /var/mail

[Ken A]

On 5/7/2010 7:00 AM, Stan Hoeppner wrote:

Charles Marcus put forth on 5/7/2010 5:29 AM:

On 2010-05-07 1:02 AM, Stan Hoeppner wrote:

I know TBird isn't the greatest IMAP client around,


Actually, its better than most (at least those with a decent GUI)...


but I think taking over
60 seconds just to download ~300 messages is way too damn long given the
hardware resources, network, and load on the client and server machines.


There is definitely something wrong.


I agree.


Do you store your profile on a remote filesystem? There is a known major
TB bug that causes it to be dog slow if your profile is not on a local
hard drive. It is apparently fixed for 3.1, and I think it even made it
into the 3.1b2 that was just released, so you might give it a try if
your profile is on a remote filesystem.


The profiles are on the local machine, which is W2K Pro SP4 all M$ patches
via auto updates, Win32 TB 3.0.4, Athlon XP 2GHz, 1GB dual channel RAM, 7.2K
rpm 120GB Seagate UDMA100.


Maybe some a/v scanning your mail?
If so, try turning it off, or switching TB to port 993 (and enable imaps 
in dovecot).


Ken


 The server is an old dual CPU 500MHz Intel box

with 384MB PC100, a new 500GB 7.2K rpm single platter WD Blue SATA drive on
a new SiI 3512 card, Intel Pro 100 NIC, Debian Lenny 5.0.4 with custom
rolled 2.6.32.9 from kernel.org source, Dovecot 1.2.11 from Lenny backports.
  For practical purposes, this is a personal server with only a single IMAP
client, load average: 0.01, 0.06, 0.03.  The only real load it gets is an
occasional kernel make, or processing a batch of digital camera photos with
imagemagick and curator.

I tested out Outlook Express 6.0, which I've never used before, but was
already on the machine as part of W2K.  There were only a couple of new
messages to grab so I couldn't test new message retrieval speed.  However,
when I clicked on a couple of IMAP folders containing over 11,000 messages
each, they transferred in about 15 seconds per folder.  It was freak'n fast.
  I was pleasantly surprised.  Granted this wasn't an apples to apples test.


TB basically seems to be pulling, or dovecot serving, only about 5
messages/sec over 100Mb ethernet, which is abysmal performance given neither
the server nor client have any load.  The messages are mostly list mail
which are at max a few kilobytes each.

I'm leaning toward a problem with TBird but I've been unable to find a bug
report that covers this, nor a forum post anywhere, etc.  The closest I've
found for "slow startup" are recommendations to compact folders.  I have no
local folders to compact.  I delete immediately and expunge on exit.


Ummm... compacting has nothing to do with 'Local Folders', it has to do
with the local mbox files that are used to store the message headers
(and bodies of downloaded messages) - and simply expunging is *not*
enough. You need to either manually compact them every now and then, or
set it to automatically compact regularly.


There are no local mbox files.  Those are only created if one sets TB to
synchronize IMAP folders to the local drive for offline use, which I do
_NOT_ do.  That defeats the whole purpose of having a nearby (network
latency and b/w wise) fast IMAP server.  If I wanted copies of all my mail
on my workstation I'd run POP.  But I don't.  Thus, I don't synchronize.

The only noteworthy TB files I have locally are .msf files in the
~\Application Data\~\ImapMail directory, one per IMAP folder on the server.
  AFAIK these are the index files TB creates of the message headers it d/l's
from the IMAP server.  I also have a couple of cache files in the other TB
profile directory ~\Local Settings\~\Cache that are rather large, one being
~50MB, the other being ~30MB, both with a current timestamp, meaning both
are actively being used.  AIUI, compacting folders in TB only affects local
mbox files, removing deleted messages, and rewriting the file to eliminate
whitespace.  In absence of this, I defrag both partitions on my workstation
disk frequently.  Even after a fresh thorough defrag, this TB startup
performance problem still exists.  AFAIK, Dovecot does something similar to
TB compacting automatically on its mbox files upon expunge.

Regardless of all the mbox and compacting talk, why would this ever affect
new message headers being served up to TB by dovecot from the /var/mail/stan
file?  Every time I exit TB /var/mail/stan gets automatically compacted by
dovecot.  When I open TB the next time, and there are 300 messages, dovecot
reads the partial headers and funnels them to TB.  Correct?  It seems TB
then spins at 100% CPU for 60+ seconds saying "Downloading header x of 300".
  When it hits ~300, then there is finally network activity as TB seems to
sort the messages into the proper IMAP folders, which is lightning quick
compared to "downloading message headers".

I don't recall having this performance issue with dovecot 1.0.15.  Just in
case it's something I nurfed in my dovecot config, here's my

Re: [Dovecot] permission denied, no logs

[Ken A]

On 4/19/2010 11:30 AM, Shane Brooks wrote:

Hi,
We have a sendmail/dovecot/squirrelmail installation that was working ok
except for deleted files showing in the list with a [d] beside it.  In
trying to fix that issue, I apparently made a new error appear in the
left navigation area:
ERROR: Could not complete request.
Query: CREATE "mail/Sent"
Reason Given: [NOPERM] Permission denied


Dovecot is telling you that it doesn't have permission to create the 
"Sent" mailbox in ~username/mail/Sent

Check the permissions on those folders.

Ken



There are no errors in dovecot, but from what i've read squirrelmail
passes these folder requests to the imap server for processing.  Can
anyone help me verify what my permissions should be?
Here is my config:
# 1.2.9: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.24-19-xen x86_64 CentOS release 5.3 (Final)
log_path: /var/log/dovecot/runtime.log
protocols: imap imaps pop3 pop3s
disable_plaintext_auth: no
login_dir: /var/run/dovecot/login
login_executable(default): /usr/libexec/dovecot/imap-login
login_executable(imap): /usr/libexec/dovecot/imap-login
login_executable(pop3): /usr/libexec/dovecot/pop3-login
mail_privileged_group: mail
mail_uid: mail
mail_gid: mail
mail_location: mbox:~:INBOX=/var/mail/%u
mail_executable(default): /usr/libexec/dovecot/imap
mail_executable(imap): /usr/libexec/dovecot/imap
mail_executable(pop3): /usr/libexec/dovecot/pop3
mail_plugin_dir(default): /usr/lib/dovecot/imap
mail_plugin_dir(imap): /usr/lib/dovecot/imap
mail_plugin_dir(pop3): /usr/lib/dovecot/pop3
lda:
   postmaster_address: postmas...@example.com
auth default:
   cache_size: 10
   debug: yes
   passdb:
 driver: pam
 args: cache_key=%u dovecot
   userdb:
 driver: passwd


Thank you very much!



Shane brooks

IT Manager



940 West Oakland Avenue
Suite A8
Oakland, Florida 34787

407-654-6076 Office
407-654-6063 Fax







--
Ken Anderson
Pacific Internet - http://www.pacific.net

Re: [Dovecot] sieve plugin - reject sends MDN. Can it 550 reject ?

[Ken A]

On 4/15/2010 4:37 PM, Scott Silva wrote:

on 4-15-2010 2:05 PM Ken A spake the following:

I'm using dovecot-1.2-sieve-0.1.15 with dovecot-1.2.11.

I have a reject test filter that looks like this:

require "reject";
if header :contains "subject" "reject" {
reject "bye";
}

Is there any way to get a "reject" filter to send back a 550 reject as
detailed in http://ietfreport.isoc.org/idref/rfc5429/#section-2.5 rather
than a MDN?

The MDN can result in blowback spam. :-(

Thanks,
Ken



Only the MTA can do that. By the time dovecot gets the message it has already
been accepted, and it is too late for a reject. You could try a milter that
runs in front of your mailserver and can reject a message during the data
phase. Once accepted, and the connection is closed, there is no way to do what
you ask.



That makes sense, duh. Default behavior is to send an immediate MDN, 
which is not what I expected. I found the setting in to get it to Defer, 
"EX_TEMPFAIL" instead (quota_full_tempfail = yes).

That makes it compatible with my existing over quota monitoring. :-)

Thanks,
Ken



--
Ken Anderson
Pacific Internet - http://www.pacific.net

[Dovecot] sieve plugin - reject sends MDN. Can it 550 reject ?

[Ken A]

I'm using dovecot-1.2-sieve-0.1.15 with dovecot-1.2.11.

I have a reject test filter that looks like this:

require "reject";
if header :contains "subject" "reject" {
reject "bye";
}

Is there any way to get a "reject" filter to send back a 550 reject as 
detailed in http://ietfreport.isoc.org/idref/rfc5429/#section-2.5 rather 
than a MDN?


The MDN can result in blowback spam. :-(

Thanks,
Ken


--
Ken Anderson
Pacific Internet - http://www.pacific.net

Re: [Dovecot] Virusscanning

[Ken A]

On 4/13/2010 6:21 AM, Andreas Schulze wrote:

Am 13.04.2010 20:37 schrieb Noel Butler:

So, you want  postfix to accept the virus, send it to dovecot's deliver
which then calls a virus scanner and finds it infected and deletes it,
that makes absolutely no sense

ACK.

but imagine:

MTA delivers a mail where the virusscanner finds nothing. Mail gets delivered.
Some time later there is a scannerupdate. Now the scanner would find a 
malicious content.

So I may instantly scan the complete mailstore each time a new pattern arrives
or scan only each accessed mail with the latest pattern. This seems smarter to 
me.



With this logic, you should virus scan at every point along the chain.
That is overkill for such a i/o intensive operation like virus scanning. 
Let the MTA scan it, and the MSA (or client's network) scan it before 
opening.

Ken



For this scenario I would like to see a concept for 
datainspection/datamodification in dovecot.
What about when dovecot would act as a milter client?
Sounds strange but the problems are the same, why not use existing solutions ?





--
Ken Anderson
Pacific Internet - http://www.pacific.net

[Dovecot] Dovecot 2.0 beta 4 errors reported

[Ken A]

Timo,

Thank you for an amazing imap/pop server.
I'm using Dovecot 2.0 beta 4 and mbox storage, and have encountered a 
couple of errors worth reporting:


 --

1. When I delete a parent folder that contains sub folders, I get this 
error message pop up in Thunderbird 3.04:



"The current command did not succeed. The mail server responded:
[CANNOT] Mailbox GUIDs are not permanent without index files."


But, the folders are deleted properly, and no error is logged on the 
server. I have the mail_log plugin currently enabled like so:

mail_plugins = quota mail_log notify
mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename
mail_log_fields = uid box msgid size


 --

2. segfault reported in the system log (unrelated to the error above):

kernel: pop3[6794]: segfault at 0028 rip 00403489
rsp 7fff20a24a20 error 4


--

Thanks,
Ken

--
Ken Anderson
Pacific Internet - http://www.pacific.net

Re: [Dovecot] IMAP timeout problem

[Ken A]

Vasantha Narayan wrote:

Hello,

We are running dovecot 1.1.2 on a Solaris 10 box.  We have been
running the same configuration for about a year without a problem.
We have a different cert for POP and IMAP.

Recently we made two changes.  The cert for IMAP expired and we had
to install a new one.  We applied some patches to the SAN disks that
hosts the mail partition.  These changes were made over a weekend and
the system was rebooted. 



Do you have any baseline benchmarks for your SAN?
What does vmstat 2 or iostat -c 2 say about i/o wait?

Ken

 There were no errors in the logs and

everything worked fine until the following Monday morning when IMAP
connections increased.   When IMAP ports started getting hammered in
the morning, IMAP clients started getting "IMAP time out" errors.
When this error occurs, IMAP freezes and IMAP clients such as,
Thunderbird and Squirrelmail  are not able to connect to the IMAP
server (both IMAP and IMAPS).   We cannot get a telnet connection to
the IMAP port or a openssl connection to the IMAPS port.  Kill -HUP
of the dovecot server solves the problem.  But again, when the usage
increases, the problem surfaces again.  There is no problem with POP
or POPS.  It continues to function normally while IMAP times out and
freezes two or three times a day.

Any clue where the problem might be?  The debug log does not have any
useful pointers.

Thanks.

Vasantha





--
Ken Anderson
Pacific Internet - http://www.pacific.net

Re: [Dovecot] sendmail smtp auth using client socket

[Ken A]

Ken A wrote:



Through pam, if you set "MECH=pam" in sasl config.
http://wiki.dovecot.org/PasswordDatabase/PAM



uh.. nevermind. got the cart before the horse there.
need more coffee..
Ken

Re: [Dovecot] sendmail smtp auth using client socket

[Ken A]

EQX wrote:


Hi list

I searched the net, but could not find something related.

Is it possible to export the client socket to sendmail for smtp auth? 
What would be the sendmail.mc configuration for this?

   #client {
  # The client socket is generally safe to export to everyone. 
Typical use

  # is to export it to your SMTP server so it can do SMTP AUTH lookups
  # using it.
  #path = /var/spool/postfix/private/auth
  #mode = 0660
  #user = postfix
  #group = postfix
#}

It can also be the other way round: can Dovecot use Cyrus SASL for user 
authentication?


Through pam, if you set "MECH=pam" in sasl config.
http://wiki.dovecot.org/PasswordDatabase/PAM

Ken



I need to use sendmail, ip based solutions like pop-before-smtp or 
poprelay are not an option.


Thanks, Mike

Re: [Dovecot] I've moved to US

[Ken A]

Timo Sirainen wrote:

I thought about saying this in v1.2.beta1 release annoouncement, but
looks like it'll take a few more days. So I've moved to Blacksburg,
Virginia and I'm now working for Mailtrust the rest of this year. Here's
some talk about it:

http://mailtrust.com/blog

(A bit stupid looking picture, but then again all my pictures seem to be
that way so I didn't bother getting a new one taken.)

And if it's not clear from that blog: Pretty much everything I do here
will be Dovecot improvements that will be released as open source.
Mailtrust has actually been paying for Dovecot features for about 3
years now.



Awesome! Now when I ask dumb questions, I will be in almost the same 
time zone as the one who answers 95% of the questions on this list!

:-)
Ken
Jonesboro, Arkansas

Re: [Dovecot] dirsize quota in sql pass db?

[Ken A]

Timo Sirainen wrote:

On Thu, 2009-02-05 at 08:38 -0600, Ken A wrote:

Timo Sirainen wrote:

On Feb 4, 2009, at 11:21 PM, Ken Anderson wrote:


Is there a way to return a per user dirsize quota from a passdb sql
lookup? If so, what is the syntax?
Yes, but only if you use prefetch userdb. Do you? If not, you'll need to 
return it from userdb.
So, prefetch userdb is all or none. My userdb is currently /etc/passwd 
to keep sendmail and other legacy stuff happy. I used fs quota, but am 
now moving INBOX to a different (faster) partition, which breaks fs 
quota reporting in things like squirrelmail.

Dovecot is 1.0.5
What would you suggest?


Since you already have your users' passwords in SQL, why not just put
the rest of the info there as well (and have it duplicated
in /etc/passwd for Sendmail)?



That makes sense.
Thanks much,
Ken

Re: [Dovecot] pop3_lock_session question

[Ken A]

Timo Sirainen wrote:

On Wed, 2009-02-04 at 14:51 -0500, Alan Ferrency wrote:

One problem which might be making this worse than it needs to be, is
the fact that mbox_lock_flock in mbox-lock.c is not using a blocking
flock(); instead, it's polling for a non-blocking lock.  This technique
can cause lock starvation, if another process is dropping the lock and
picking it back up again frequently: other processes will only see the
lock as being available if they happen to poll for the lock at just
the right instant.

A better technique to use here, if it's adequately cross-platform,
would be to set an alarm() for the max_wait_time, and use a blocking
flock().  If the alarm times out and you don't have a lock, it's a
timeout.  In the meantime, you're guaranteed to eventually get the
lock, if it is dropped.


That's what Dovecot does elsewhere. I don't really know why I'm using
non-blocking flock() calls. I guess I should fix that.


That said: I'm not sure whether this will solve our problem in
practice.


Probably not.


Why doesn't this happen with imap?  Why can't we make pop3 do what
imap does?  Even if it's inefficient, it's better than hanging all
incoming mail delivery while deliver eats up our local concurrency
limits.


IMAP unlocks mbox after each command is done. But POP3 clients typically
just run RETR, RETR, RETR, .. so unlocking + locking again later is just
extra work that slows things down. I guess there could be a timeout that
if no RETR has been run for a few seconds it unlocks the mailbox.

But I've never before heard POP3 clients behaving that way, so I'd like
to know what exactly are they doing. Are they not sending anything? Are
they NOOPing? I don't see any reason for them to be doing either..


We see it (procmail waiting on pop clients doing nothing) when a 
connection slows - sometimes to a crawl - on rural U.S. phone lines. 
Dovecot usually disconnects them after 10 minutes if the connection 
stops, but sometimes that process can drag on for a while. I've watched 
these using tcpdump. Analog modems can be quite persistent.

Ken

Re: [Dovecot] dirsize quota in sql pass db?

[Ken A]

Timo Sirainen wrote:

On Feb 4, 2009, at 11:21 PM, Ken Anderson wrote:


Is there a way to return a per user dirsize quota from a passdb sql
lookup? If so, what is the syntax?


Yes, but only if you use prefetch userdb. Do you? If not, you'll need to 
return it from userdb.


So, prefetch userdb is all or none. My userdb is currently /etc/passwd 
to keep sendmail and other legacy stuff happy. I used fs quota, but am 
now moving INBOX to a different (faster) partition, which breaks fs 
quota reporting in things like squirrelmail.

Dovecot is 1.0.5
What would you suggest?
Thanks,
Ken




maybe "select quota as userdb_quota"?

..and in mysql table, quota column would be:
"quota=dirsize:storage=10240" ?


"dirsize:storage=10240" actuall. Or more easily keep the quota number in 
SQL and use:


select concat('dirsize:storage=', quota) as userdb_quota


I'm using mbox, with /etc/passwd userdb, and mysql passdb.


What Dovecot version?

Re: [Dovecot] Backing Up

[Ken A]

Calvin Gordon wrote:
I use the tar/bzip method, and have been wondering about the rsync.  All 
my users have system accounts on the dovecot server, and use Maildir 
format.  If i rsync the mail to another box where the users do not have 
system accounts, will the ownerships/ permissions etc. be goofed up ?


Correctly, or incorrectly, I've been using tar to preserve all that 
information.


rsync preserves all that too, but you should preserve uid->username and 
gid->groupname mappings too, otherwise all that information is not as 
useful. Saving the password files is usually sufficient, assuming you 
are doing backups for disaster recovery, and not just for the occasional 
restore after an "oops, I deleted all my mail!" phonecall.


rsnapshot is nice too. It uses rsync and hard links to make as many 
snapshots of the filesystem as you like. This creates many 'restore 
points' with total disk usage being just over what a single full backup 
would take.


Ken



Cal Gordon

Sotiris Tsimbonis wrote:

Scott Silva wrote, On 10/30/2008 12:34 AM:

on 10-29-2008 3:18 PM Dave McGuire spake the following:

On Oct 29, 2008, at 5:32 PM, Arkadiusz Miskiewicz wrote:

What is the best way to do a (server-side) backup of all mail in a
user's mail?

I usually just rsync the /home directories to another server. The
inital sync
can take a while, but it gets faster after there is a base to work
from.

   ...and it's much less painful if you're using maildir instead of
mbox!

Not for rsyncing. Tons of small files means much slower rsync.
  Due to connection turnaround latency, I assume?  (I've never 
looked at

the rsync protocol)  If that's the case, then I stand very much
corrected, thank you.  I was going from the same logic regarding mbox
vs. maildir in the context of backups.  One new message delivered and a
400MB mail spool gets backed up again..

  -Dave


Rsync adds some latency as it indexes and compares files on both ends.
Obviously it would take more time to compare 40,000 1K files then 
1000 40K
files even though the data size is similar. It would still be better 
than
tar/bzip/scp which has to compress everything and transfer the lot 
every time.




Maildirsync it an "Online synchronizer for Maildir-format mailboxes"
See http://hacks.dlux.hu/maildirsync/

Sot.






--
Ken Anderson
Pacific.Net

Re: [Dovecot] Pop3 Problem

[Ken A]

Rajiv Gore wrote:

Hi
I want to use Dovecot as POP3 server.
Config file is changed to indicate IMAP , POP3
On starting Dovecot I get an error indicating
"listen(110) failed: Address already in use"
lsof -i :110 indicates Xinetd is using this port.
How do I get POP3 to use this port.
Regards


cd /etc/xinetd.d
ls
see any pop3 server there?
vi [name of pop3 server you just spotted]
change disable=false to disable=true
/etc/init.d/xinetd restart



--
Ken Anderson
Pacific.Net

Re: [Dovecot] Quota Issues

[Ken A]

Peter Fraser wrote:

Hi All

Is there a way to test to see if the dovecot quota settings are being
applied?


http://www.google.com/search?q=telnet+IMAP+QUOTA


I'm doing this as part of troubleshooting.
I have set up dovecot + quotas  with squirrelmail and I am not seeing the
quota graphs in squirrelmail but I am not getting any errors.
Could someone please help me with this?

THanks.





--
Ken Anderson
Pacific.Net

Re: [Dovecot] POP3 locking

[Ken A]

Timo Sirainen wrote:

On Jul 31, 2008, at 8:10 PM, Mark Sapiro wrote:


On Wed, Jul 30, 2008 at 06:05:27PM +0200, Kai Schaetzl wrote:


but it seems to be off by default. From my dovecot.conf (1.07):

#pop3_lock_session = no

I think it should be on by default.



The problem is if you have an MUA/MDA retrieving lots of messages
over a potentially slow connection, the mailbox can remain locked
for a very long time for a read-only operation.


Or before when dialups were more common, if the modem disconnected you'd 
have to wait several minutes until the previous connection timed out.


Still very common in rural USA. We see occasional pop3 locks that last 
hours while someone on a 26.4Kbps connect downloads mail. rfc1939 says 
it should be locked. I do understand why it's not default in dovecot 
though, since Dovecot is primarily an IMAP server, and the idea of 
locking a mailbox for hours is probably a bit repulsive to the 
developer. :-)


Ken




The problem that requires doing it is basically a client problem.
Namely, some clients may get confused if in the middle of a series
of RETR commands, they get a mailbox locked reply.


This should never happen with Dovecot. But what could happen is that the 
message got expunged and the RETR would fail because of that.





--
Ken Anderson
Pacific.Net

Re: [Dovecot] mbox purge routine?

[Ken A]

Rich Winkel wrote:

Can someone direct me to a unix utility for purging old messages from
mbox's?  It seems entourage isn't doing it.

Thanks,
Rich



archmbox (perl command line tool for mboxes) is my chosen tool for that 
these days. It's on the sourceforge site.


--
Ken Anderson
Pacific.Net

[Dovecot] login_processes_count & login_max_processes_count question

[Ken A]

re:


# Maximum number of login processes to create. The listening process count
# usually stays at login_processes_count, but when multiple users start logging
# in at the same time more extra processes are created. To prevent fork-bombing
# we check only once in a second if new processes should be created - if all
# of them are used at the time, we double their amount until the limit set by
# this setting is reached.
#login_max_processes_count = 128


If the server is attacked by a password guessing routine, or just gets 
very busy, and login process count reaches 128, will the created 
processes ever get killed, or will dovecot continue to run 128 processes 
until it's restarted?


Thanks,
Ken


--
Ken Anderson
Pacific.Net

Re: [Dovecot] trouble deleting folders that only contain folders

[Ken A]

Ken A wrote:

Ken A wrote:

Timo Sirainen wrote:

On Fri, 2007-10-05 at 21:17 -0500, Ken A wrote:

x list "" dir
* LIST (\Noselect \HasChildren) "/" "dir"
x OK List completed.
x delete dir
x NO Mailbox isn't selectable: dir


That looks like it's trying to open the mailbox before deleting it. What
plugins do you use?



quota imap_quota


In tracking this down, I found that by commenting the line below in 
quota-storage.c, the problem disappears, but I'm not sure what all of 
the implications are for the quota tracking. I wasn't able to figure out 
why quota_mailbox_delete() wasn't working correctly for me.


// Don't override mailbox_delete, so we can delete folders.
// storage->v.mailbox_delete = quota_mailbox_delete;

I noticed there was a quota rewrite patch for 1.0.2. Do you think I 
should give this a try for 1.0.5 ?


Thanks,

Ken






Confirmed that it has something to do with plugins.
If I comment out "mail_plugins = quota imap_quota" line in imap section, 
I can delete folders.


x delete dir
x OK Delete completed.

My quota is a simple fs quota, config taken from
http://wiki.dovecot.org/Quota/FS

protocol imap {
  mail_plugins = quota imap_quota
}
plugin {
  quota = fs
}


The quota usage is being reported correctly in squirrelmail, and works 
properly on the system (fedora core 7). Any ideas what might be causing 
this?


Thanks,
Ken





--
Ken Anderson
Pacific.Net

Re: [Dovecot] trouble deleting folders that only contain folders

[Ken A]

Ken A wrote:

Timo Sirainen wrote:

On Fri, 2007-10-05 at 21:17 -0500, Ken A wrote:

x list "" dir
* LIST (\Noselect \HasChildren) "/" "dir"
x OK List completed.
x delete dir
x NO Mailbox isn't selectable: dir


That looks like it's trying to open the mailbox before deleting it. What
plugins do you use?



quota imap_quota



Confirmed that it has something to do with plugins.
If I comment out "mail_plugins = quota imap_quota" line in imap section, 
I can delete folders.


x delete dir
x OK Delete completed.

My quota is a simple fs quota, config taken from
http://wiki.dovecot.org/Quota/FS

protocol imap {
  mail_plugins = quota imap_quota
}
plugin {
  quota = fs
}


The quota usage is being reported correctly in squirrelmail, and works 
properly on the system (fedora core 7). Any ideas what might be causing 
this?


Thanks,
Ken


--
Ken Anderson
Pacific.Net

Re: [Dovecot] trouble deleting folders that only contain folders

[Ken A]

Timo Sirainen wrote:

On Fri, 2007-10-05 at 21:17 -0500, Ken A wrote:

x list "" dir
* LIST (\Noselect \HasChildren) "/" "dir"
x OK List completed.
x delete dir
x NO Mailbox isn't selectable: dir


That looks like it's trying to open the mailbox before deleting it. What
plugins do you use?



quota imap_quota

--
Ken Anderson
Pacific.Net

Re: [Dovecot] trouble deleting folders that only contain folders

[Ken A]

Timo Sirainen wrote:

On Fri, 2007-10-05 at 16:51 -0500, Ken A wrote:

Hi,

I'm using dovecot 1.0.5 with mbox storage. Users can't delete folders 
that are created to 'only hold other folders'. I can create these 
folders with no problem. They are directories in the file system. Normal 
folders (which are mbox files) are deleted with no problem. I've tried 
with both squirrelmail and Thunderbird, and what happens when a delete 
is requested, is that all files within the folder's dir tree are deleted 
and the client is unsubscribed, but the directory tree remains.


I think this is a client bug. See what commands the client sends. Seems
to work right with me:

x create dir/
x OK Create completed.
x list "" dir
* LIST (\Noselect \HasChildren) "/" "dir"
x OK List completed.
x delete dir
x OK Delete completed.
x list "" dir
x OK List completed.



I get this, from telnet:

x create dir/
x OK Create completed.
x list "" dir
* LIST (\Noselect \HasChildren) "/" "dir"
x OK List completed.
x delete dir
x NO Mailbox isn't selectable: dir
x list "" dir
* LIST (\Noselect \HasChildren) "/" "dir"
x OK List completed.


I must have something misconfigured.
mail_location: 
mbox:/var/spool/mail/folders/%u:INBOX=/var/spool/mail/%u:INDEX=/var/spool/dovecot_indexes/%u


Thanks for any assistance,
Ken


--
Ken Anderson
Pacific.Net

[Dovecot] trouble deleting folders that only contain folders

[Ken A]

Hi,

I'm using dovecot 1.0.5 with mbox storage. Users can't delete folders 
that are created to 'only hold other folders'. I can create these 
folders with no problem. They are directories in the file system. Normal 
folders (which are mbox files) are deleted with no problem. I've tried 
with both squirrelmail and Thunderbird, and what happens when a delete 
is requested, is that all files within the folder's dir tree are deleted 
and the client is unsubscribed, but the directory tree remains.


Any ideas?

Thanks,
Ken

--
Ken Anderson
Pacific.Net

Re: [Dovecot] dovecot 1.05 err: Reason Given: Mailbox isn't a valid mbox file

[Ken A]

Timo Sirainen wrote:

On Fri, 2007-09-28 at 17:04 -0500, Ken A wrote:

removing the () from the first line of the mbox fixed it.

The line:

 From [EMAIL PROTECTED] () Mon Jun 25 11:20:33 2007

Is this a valid line?


There isn't an mbox standard, so it may or may not be valid. I've made
Dovecot be pretty strict about what the From lines may look like.

What generated this line? Although it gives this error at the beginning
of file, if it happens elsewhere in the mbox file no errors are given,
and instead the mail becomes part of the previous mail.



I believe it's this line:
print MAIL "From: $Config{'email'} ($Config{'realname'})\n";
from this evil perl script:
http://www.scriptarchive.com/download.cgi?s=formmail&c=txt&f=FormMail%2Epl

This one script has caused more grief... :-(

Ken


--
Ken Anderson
Pacific.Net

[Dovecot] dovecot 1.05 err: Reason Given: Mailbox isn't a valid mbox file

[Ken A]

ERROR:
ERROR: Could not complete request.
Query: SELECT "Mailbox"
Reason Given: Mailbox isn't a valid mbox file


removing the () from the first line of the mbox fixed it.

The line:

From [EMAIL PROTECTED] () Mon Jun 25 11:20:33 2007

Is this a valid line?

--
Ken Anderson
Pacific.Net

Re: [Dovecot] deliver w/quotas - MDN after accept mail?

[Ken A]

Ken A wrote:

thanks! -e did it.


correction.. sendmail says EX_NOPERM is a permanent error, and issues 
it's own DSN immediately. "dsn=5.0.0, stat=Insufficient permission".

Ken



Ken

pod wrote:

"KA" == Ken A <[EMAIL PROTECTED]> writes:


KA> I'm testing deliver with sendmail and fs quotas.

KA> On an over quota condition, deliver accepts the mail, deletes it,
KA> then issues a MDN immediately.

KA> Is there a way to get deliver to soft fail like procmail does 
with

KA> a 400 error and queue the mail, then let sendmail handle the MDN,
KA> following it's "confTO_QUEUEWARN" and "confTO_QUEUERETURN" ?

Last time I looked at deliver it seemed like there were very few
conditions it considered as a temporary failure and thus for it to return
EX_TEMPFAIL.  I also was considering over-quota conditions (also
filesystem quotas).

The '-e' option to deliver will, possibly by accidental side-effect, 
avoid

calling the deliver-generated bounce code.  Instead deliver will write
some error text on stderr and exit EX_NOPERM.

The code in question is src/deliver/deliver.c lines 810 -- 835 which
occurs right after attempting to save the message (i.e. ret is the return
code from the save attempt).  The '-e' option is what sets
stderr_rejection.



Tried with -e, but sendmail says EX_NOPERM is a permanent error, and 
issues it's own DSN immediately. "dsn=5.0.0, stat=Insufficient 
permission". :-(


Ken




if (ret < 0) {
const char *error, *msgid;
bool syntax, temporary_error;
int ret;

error = mail_storage_get_last_error(storage, &syntax,
&temporary_error);
if (temporary_error)
return EX_TEMPFAIL;

msgid = mail_get_first_header(mail, "Message-ID");
i_info("msgid=%s: Rejected: %s",
   msgid == NULL ? "" : str_sanitize(msgid, 80),
   str_sanitize(error, 512));

/* we'll have to reply with permanent failure */
if (stderr_rejection) {
fprintf(stderr, "%s\n", error);
return EX_NOPERM;
}
ret = mail_send_rejection(mail, destination, error);
if (ret != 0)
return ret < 0 ? EX_TEMPFAIL : ret;
/* ok, rejection sent */
}

As Timo has said elsewhere "deliver could use a rewrite some day..".









--
Ken Anderson
Pacific.Net

Re: [Dovecot] deliver w/quotas - MDN after accept mail?

[Ken A]

thanks! -e did it.
Ken

pod wrote:

"KA" == Ken A <[EMAIL PROTECTED]> writes:


KA> I'm testing deliver with sendmail and fs quotas.

KA> On an over quota condition, deliver accepts the mail, deletes it,
KA> then issues a MDN immediately.

KA> Is there a way to get deliver to soft fail like procmail does with
KA> a 400 error and queue the mail, then let sendmail handle the MDN,
KA> following it's "confTO_QUEUEWARN" and "confTO_QUEUERETURN" ?

Last time I looked at deliver it seemed like there were very few
conditions it considered as a temporary failure and thus for it to return
EX_TEMPFAIL.  I also was considering over-quota conditions (also
filesystem quotas).

The '-e' option to deliver will, possibly by accidental side-effect, avoid
calling the deliver-generated bounce code.  Instead deliver will write
some error text on stderr and exit EX_NOPERM.

The code in question is src/deliver/deliver.c lines 810 -- 835 which
occurs right after attempting to save the message (i.e. ret is the return
code from the save attempt).  The '-e' option is what sets
stderr_rejection.



Tried with -e, but sendmail says EX_NOPERM is a permanent error, and 
issues it's own DSN immediately. "dsn=5.0.0, stat=Insufficient 
permission". :-(


Ken




if (ret < 0) {
const char *error, *msgid;
bool syntax, temporary_error;
int ret;

error = mail_storage_get_last_error(storage, &syntax,
&temporary_error);
if (temporary_error)
return EX_TEMPFAIL;

msgid = mail_get_first_header(mail, "Message-ID");
i_info("msgid=%s: Rejected: %s",
   msgid == NULL ? "" : str_sanitize(msgid, 80),
   str_sanitize(error, 512));

/* we'll have to reply with permanent failure */
if (stderr_rejection) {
fprintf(stderr, "%s\n", error);
return EX_NOPERM;
}
ret = mail_send_rejection(mail, destination, error);
if (ret != 0)
return ret < 0 ? EX_TEMPFAIL : ret;
/* ok, rejection sent */
}

As Timo has said elsewhere "deliver could use a rewrite some day..".






--
Ken Anderson
Pacific.Net

[Dovecot] deliver w/quotas - MDN after accept mail?

[Ken A]

I'm testing deliver with sendmail and fs quotas.

On an over quota condition, deliver accepts the mail, deletes it, then 
issues a MDN immediately.


Is there a way to get deliver to soft fail like procmail does with a 400 
error and queue the mail, then let sendmail handle the MDN, following 
it's "confTO_QUEUEWARN" and "confTO_QUEUERETURN" ?


Thanks,
Ken

--
Ken Anderson
Pacific.Net

Re: [Dovecot] o/s tuning for imap

[Ken A]

Kyle Wheeler wrote:

On Thursday, September  6 at 02:59 PM, quoth Ken A:
We found that on our server, *not* using imapproxy improved our 
performance. We used to use imapproxy to great effect when we were 
using BincIMAP, but Dovecot is so darn fast (and caches its own 
authentication) that all imapproxy added was additional inter-process 
communication (translation: slower than just using Dovecot alone).


My understanding is that webmail clients like squirrelmail open, then 
close connections on each http transaction that requires a connection 
to the imap server, so imapproxy's caching of connections saves you 
having to re-open connections to the backend server. 


Yes it does, you are exactly correct. HOWEVER, here's something to keep 
in mind: is that really a problem? Consider, for example, that you're 
not saving any "making a new connection" overhead, because your webmail 
client is still making a new connection to the imapproxy. So, connection 
setup and teardown is still there. If your backend is on another machine 
and your proxy is on localhost, what you're really doing is moving the 
setup and teardown from being done over the network to being done over 
the loopback interface. If your network is particularly busy, this *can* 
be a win, but it may not provide much benefit if the connection from 
webmail server to imap server is a fast ethernet connection with not 
much else to do.


The other thing that an imapproxy changes is it means that the webmail  
client doesn't have to re-authenticate every time; it caches the string 
that the webmail client sent before, and if there's a connection open 
that was approved with those authentication strings, you get it. Thus 
something that might require some hashing and possibly an ldap-lookup 
gets turned into a simple strcmp(). BUT dovecot does this already, with 
its authentication server. So imapproxy isn't providing much of a win 
there either (with dovecot).


You have to consider what the proxy is actually saving you, and whether 
it's worth it. The backend server isn't typically connection starved 
(and if it is, then you haven't configured it properly).


That's essentially why my original question included "what about 
time_wait", since I was concerned that squirrelmail could leave a LOT 
of connections in a TIME_WAIT state.


That's a fair observation, but it's not going to leave any fewer 
connections in such a state if it's connecting to the proxy server 
rather than dovecot.


Personally, on my squirrelmail installation, most users have about five 
or so connections in TIME_WAIT while they're actively using webmail. 
Thus far that hasn't been a problem, but I don't have a heavily loaded 
webmail service.


~Kyle



What I've found is that using squirrelmail with imapproxy to dovecot is 
is faster because TLS connections are cached.


Without TLS, squirrelmail is faster without imapproxy. So, if 
squirrelmail and dovecot are on the same box, imapproxy will only slow 
things down.


So, imapproxy saves more than it costs by caching TLS connections, and 
it's a lot cheaper than a couple of accelerator cards.


Ken


--
Ken Anderson
Pacific.Net

Re: [Dovecot] o/s tuning for imap

[Ken A]

Kyle Wheeler wrote:

On Tuesday, September  4 at 08:26 PM, quoth Russell E. Meek:
OS related tweaks, probably not.  However you could utilize a imap 
proxy such as up-imapproxy which if using FreeBSD is in ports.


Visit: http://www.imapproxy.org/ to learn more.

This should relieve the load on Dovecot.


We found that on our server, *not* using imapproxy improved our 
performance. We used to use imapproxy to great effect when we were using 
BincIMAP, but Dovecot is so darn fast (and caches its own 
authentication) that all imapproxy added was additional inter-process 
communication (translation: slower than just using Dovecot alone).


~Kyle


My understanding is that webmail clients like squirrelmail open, then 
close connections on each http transaction that requires a connection to 
the imap server, so imapproxy's caching of connections saves you having 
to re-open connections to the backend server. That's essentially why my 
original question included "what about time_wait", since I was concerned 
that squirrelmail could leave a LOT of connections in a TIME_WAIT state.


Thanks for your other suggestions,
Ken


--
Ken Anderson
Pacific.Net

Re: [Dovecot] securing dovecot proxy connections

[Ken A]

WJCarpenter wrote:

ka> per another current thread (o/s tuning for imap), I've installed
ka> imapproxy, and it supports starttls to the backend imap server. It
ka> doesn't use encryption on the incoming connections though, since
ka> they are presumably from localhost (squirrelmail).  Ken

That's an interesting thought.  Have you actually gotten its STARTTLS
to work?  I tried it a couple days ago with no luck, but maybe I just
didn't try hard enough.


I had a bit of trouble with it as well. I set it to connect to the 
dovecot on 993 and listen on 143 locally. That had an odd effect. It 
connected to dovecot, but failed to LISTEN locally, and nothing was 
logged about this failure! After trying various other things, I changed 
the settings to 143 for both and it worked. Then I set "force tls = 
yes", pointed the config options at the bottom of the config file to 
valid cert,ca,key, etc.. and yes, it works great. tcpdump verifies that 
it's using TLS now.


Ken


--
Ken Anderson
Pacific.Net

Re: [Dovecot] securing dovecot proxy connections

[Ken A]

WJCarpenter wrote:

The wiki 
page says this: "The connections created to the destination server
can't be TLS/SSL encrypted.".

Hrmm.  Right now, with perdition, I'm forcing the use of STARTTLS on
the internal connections.  I'd just as soon get rid of perdition (to
have one less moving part in my architecture), but I need the secure
connections.  


Is there a way to configure dovecot's internal proxy connections to
use STARTTLS or some other SSL/TLS level of security?  (Without a
bunch more research, I don't know what the interaction is between the
real client, the dovecot proxy, and the destination server.)


per another current thread (o/s tuning for imap), I've installed 
imapproxy, and it supports starttls to the backend imap server. It 
doesn't use encryption on the incoming connections though, since they 
are presumably from localhost (squirrelmail).

Ken


--
Ken Anderson
Pacific.Net

[Dovecot] o/s tuning for imap

[Ken A]

I'm switching from a pop3 only dovecot install to a pop3/imap install 
and I'm wondering how many connections every 100 'normal' imap users 
might have/keep open? I'm wondering if I need to tweak any o/s related 
things, like time_wait, etc. Any pointers would be greatly appreciated.


Thanks,
Ken A.

Re: [Dovecot] telnet to dovecot not creating mbox

[Ken A]

Andy Fadich wrote:

Hi,
 
Whenever I create a new email user, telnet fails until an email is sent to

that user to create his mbox.  This is the error that it gives. Anyone know
how to fix this?  I could create the email account script to touch the users
mbox, but I havn't needed to do this in the past with ipop3d and dovecot on
a different server.
 
 
[EMAIL PROTECTED] log]# telnet localhost 110

Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
+OK Dovecot ready.
user testact
+OK
pass
+OK Logged in.
-ERR Internal error occurred. Refer to server log for more information.
[2007-07-13 10:37:15]
Connection closed by foreign host.

 
Jul 13 10:33:43 mail dovecot: pop3-login: Login: user=,

method=PLAIN, rip=:::127.0.0.1, lip=:::127.0.0.1, secured
Jul 13 10:33:43 mail dovecot: POP3(testlol): open(/var/spool/mail/testact,
O_CREAT) failed: Permission denied
Jul 13 10:33:43 mail dovecot: POP3(testlol): access() failed with mbox file
/var/spool/mail/testact: No such file or directory
Jul 13 10:33:43 mail dovecot: POP3(testlol): stat() failed with mbox file
/var/spool/mail/testact: No such file or directory
Jul 13 10:33:43 mail dovecot: POP3(testlol): Couldn't init INBOX: Internal
error occurred. Refer to server log for more information. [2007-07-13
10:33:43]
Jul 13 10:33:43 mail dovecot: POP3(testlol): Mailbox init failed top=0/0,
retr=0/0, del=0/0, size=0
 
Andy




mail_extra_groups = mail
?
--
Ken Anderson
Pacific.Net