[Dovecot] stats: duration of sessions, commands

2013-03-31 Thread Lutz Preßler
Hello Timo,

I would like to have reset_timestamp (or another field) in stats
dump output of levels session and command. Both session duration
and individual command execution times would be interesting
data to have.

Greetings,
   Lutz


Re: [Dovecot] Dovecot 2.2rc3 Client Cert Auth and Webmail -> auth_ssl_require_client_cert problem [solved]

2013-03-31 Thread Christian Felsing
Thank you, works now with 'Y' as nopassword :-)

best regards
Christian

Am 31.03.2013 17:16, schrieb Timo Sirainen:
> On 31.3.2013, at 17.38, Christian Felsing  wrote:
> 
>> There were log entries regarding that problem:
> 
> Ah, you were using PostgreSQL and I tested MySQL. They are handled somewhat 
> differently. This should fix it: 
> http://hg.dovecot.org/dovecot-2.2/rev/37cd62516b37



Re: [Dovecot] Incorrect response when opening read-only mailboxes with per-user flags?

2013-03-31 Thread Jake Johnson
unsubscribe


On Wed, Mar 27, 2013 at 1:16 PM, Guido Berhoerster <
guido+dovecot@berhoerster.name> wrote:

> * Timo Sirainen  [2013-03-27 20:27]:
> > On 27.3.2013, at 13.41, Guido Berhoerster <
> guido+dovecot@berhoerster.name> wrote:
> >
> > > After some debugging on the protocol level I've finally found the
> > > problem with per-user seen flags in read-only mailboxes. Dovecot
> > > actually does allow storing the seen flag on messages but AFAICS
> > > it does not communicate that correctly to clients, this is what
> > > its response to selecting a read-only mailbox with "lrs" ACL in
> > > effect for the logged in user looks like:
> > >
> > > 2 SELECT "Public/Test"
> > ..
> > > 2 OK [READ-ONLY] Select completed.
> > >
> > > Shouldn't PERMANENTFLAGS contain \Seen here if per-user seen
> > > flags are enable and should it even be tagged READ-ONLY at all in
> > > this case? At least clients such as Python imaplib, IMAPClient,
> > > or Roundcube will open such mailboxes with EXAMINE and thus do
> > > not allow the modification of flags.
> >
> > Are you still testing on v2.0? It could be a bug in it.
>
> Yes, this is 2.0.9 from RHEL6.
> --
> Guido Berhoerster
>


Re: [Dovecot] dsync mirror to convert from mbox (1.2x) to mdbox (2.2)

2013-03-31 Thread Jake Johnson
unsubscribe


On Thu, Mar 21, 2013 at 3:31 PM, Ken A  wrote:

> I'm converting mail from mbox to mdbox.
> I've copied mail over to a new system, and now I'm running dsync like this:
>
> dsync -u username mirror
> mbox:/folder_location/username:INBOX=/inbox_location/username
>
> That seemed to work, with a couple minor errors:
>
> > dsync(username): Warning: Our dotlock file /mail_location/username.lock
> was deleted (locked 20 secs ago, touched 20 secs ago)
> > dsync(username): Error: file_dotlock_delete() failed with mbox file
> /mail_location/username: No such file or directory
>
> Do I also need to specify where dovecot's indexes are for the old mbox
> files, or is this not needed for mbox conversions (when they are the
> source)?
>
> Thanks,
> Ken A.
>


Re: [Dovecot] Dovecot 2.2rc3 Client Cert Auth and Webmail -> auth_ssl_require_client_cert problem

2013-03-31 Thread Jake Johnson
unsubscribe


On Wed, Mar 27, 2013 at 1:49 AM, Christian Felsing <
hostmas...@taunusstein.net> wrote:

> Hello,
>
> I would like to set up a Dovecot based mail system which uses X.509
> Client Certificates for authentication. A webmail system based on Horde5
> should use Dovecot as backend.
>
> For now Dovecot works with client certificates issued by my CA and Horde
> authenticates also with same client certs. Due to protocol it is
> impossible to use client certs presented by user to Horde for
> authentication at Dovecot, so Horde should be allowed to authenticate
> itself without or an arbitrary password to Dovecot. Horde and Dovecot
> are running in same protected LAN.
>
> Unfortunately Dovecot does not support different authentication methods
> on different IP addresses or ports. This does not work:
>
> remote 192.168.116.28/32 {
>   auth_ssl_require_client_cert = no
>   auth_ssl_username_from_cert = yes
>   disable_plaintext_auth = no
>   ssl = yes
>
> }
>
> Result is "doveconf: Fatal: Error in configuration file
> /opt/dovecot-2.2.rc3/etc/dovecot/conf.d/10-auth.conf line 103: Auth
> settings not supported inside local/remote blocks:
> auth_ssl_require_client_cert"
>
> Replacing "auth_ssl_require_client_cert = no" by "ssl_verify_client_cert =
> no" does not yield in an error, but it does nothing, Dovecot still
> insists for a client certificate.
>
> I afraid that I am trapped by this problem:
>
> http://dovecot.2317879.n4.nabble.com/Problem-with-requiring-client-certificates-for-external-connections-tp475.html
>
> Is there any way to turn off client certs for specific local or remote
> IP addresses?
>
> best regards
> Christian
>


Re: [Dovecot] Dovecot METADATA support

2013-03-31 Thread Timo Sirainen
On Wed, 2013-03-27 at 21:25 +0200, Timo Sirainen wrote:

> The metadata plugin started with implementing the IMAP commands and
> kind of ignored the difficult parts of fitting it in with everything.
> The stuff that is in Dovecot now handles most of the difficult parts,
> but the IMAP commands are missing. There are still two somewhat
> difficult things missing:

So, here are some plans for them:

>  * Where/how to keep track of metadata quota to prevent abuse? Maybe
> some limits to number of entries, max. bytes total in entries, max.
> bytes per entry. Although the last one only is useful for those
> backends that can't handle large entry sizes.

I guess there should be a total number of metadata items and total bytes
of metadata, similar as to mails. Possibly it could even share the same
quotas by default.

Whenever metadata item is updated, its key is logged to
dovecot.index.log among (in future version) the value's length. These
can be used to update quota and minimize locking waits:

 - remember current transaction log offset
 - read existing modified keys' value lengths
 - mailbox metadata lock
 - update new keys' values, remember their lengths
 - commit transaction
 - mailbox metadata unlock
 - scan transaction log for changes to same keys, replace old key length
with latest seen key length
 - increase quota with (new length - previous length)

Would be nice to get rid of the locking, but I guess it's not possible
without causing a race condition (S1, S2 = session 1/2):

S1: read 10
S1: write 25
S2: read 25
S2: write 30

a) S1 committed first:
old=10, new=30 -> quota +20
S1: 25-10 = +15
S2: 30-25 = +5

b) S2 committed first (which can't happen with locking):
old=10, new=25 -> quota +15
S1: 25-30 = -5
S2: 30-25 = +5 [30-10=+20 would be correct..]

>  * Currently it's configured via mail_attribute_dict, which can e.g.
> point to a file in user's home directory. How to handle shared
> metadata in shared folders? Should userA just be accessing userB's
> dict file? Where are metadata stored for public folders that have no
> owner? Should there be two dict settings for private and public dict?
> (The last, or maybe all, would be a non-issue if I had already
> implemented my idea of having only one dict configuration where
> different prefixes could be mapped to completely different
> locations/backends/etc.)

METADATA RFC says that shared metadata is required, while private
metadata isn't required. So most of the metadata is probably marked as
shared, even for mailboxes without any ACLs. So for efficiency most of
the metadata should be stored in each user's own metadata database
rather than one huge shared database. So that's why mail_attribute_dict
must be used for private and shared metadata for your user's own
mailboxes. That also forces using the same for shared mailboxes.

For public mailboxes there is no owner user, so I guess there needs to
be a new setting mail_attribute_public_dict. I was also considering a
per-namespace mail_attribute_dict, but maybe that's not worth the
trouble for now. In any case in future the idea is to allow all mail
settings to be overridden by putting them inside namespace {}.




Re: [Dovecot] v2.2 getting NULL pointer reference with shared namespace in mailbox_tree?

2013-03-31 Thread Timo Sirainen
On Fri, 2013-03-22 at 18:34 +0100, Walter Steiner wrote:
> > You most likely want to have subscriptions=no for your shared namespace. 
> > Also you most likely want to enable ACL plugin.
> 
> Yes - thank you for your comment!  The ACLs I had configured before are
> not enabled in this minimalistic configuration because of crashes when
> running doveadm backup -R ... imapc: with ACLs enabled.
> 
> You wrote that you are continuing debugging "my problem".  Regardless
> of this, should I investigate / file this ACL (related) bug?
> 
> With todays nightly:
> 
> dsync(wsunp...@iai.uni-bonn.de): Panic: file imapc-list.c: line 199 
> (imapc_list_get_vname): assertion failed: (strncmp(prefix, storage_name, 
> prefix_len) == 0 && storage_name[prefix_len] == list->sep)

Oh, and this would be good to solve too. What does it say with gdb:

gdb dsync core
fr 5
p prefix
p storage_name
p prefix_len
p list.sep
p list.ns.prefix




Re: [Dovecot] v2.2 dsync

2013-03-31 Thread Timo Sirainen
On 22.3.2013, at 10.47, Walter Steiner  wrote:

> doveadm backup -R -u ... imapc:
> => mailboxes are created and some/many messages but not all messages
>   are copied from the origin cyrus mailbox to the dovecot box.
>   (i.e. the first consecutive 233 out of 523 are okay)
> 
> All of the missing messages are logged:
> => dsync(...): Error: Mailbox ...: Remote didn't send mail UID=...
> (references to this error seen on the list in Jan. with older versions)

Finally fixed: http://hg.dovecot.org/dovecot-2.2/rev/800836bd8f2e

Also if there are any similar bugs left, it no longer exits with success: 
http://hg.dovecot.org/dovecot-2.2/rev/0eef38d78069



Re: [Dovecot] Dovecot 2.2rc3 Client Cert Auth and Webmail -> auth_ssl_require_client_cert problem

2013-03-31 Thread Timo Sirainen
On 31.3.2013, at 17.38, Christian Felsing  wrote:

> There were log entries regarding that problem:

Ah, you were using PostgreSQL and I tested MySQL. They are handled somewhat 
differently. This should fix it: 
http://hg.dovecot.org/dovecot-2.2/rev/37cd62516b37

> 
> 
> Mar 25 11:05:21 dovecot dovecot: auth: Debug: client in:
> AUTH#0111#011PLAIN#011service=imap#011secured#011valid-client-cert#011sessi
> on=J8pV8bzYIACwxigG#011cert_username=u...@example.net#011lip=192.168.200.22#011rip=192.168.200.6#011lport=993#011rport=8480
> Mar 25 11:05:21 dovecot dovecot: auth: Debug: client passdb out:
> CONT#0111#011
> Mar 25 11:05:21 dovecot dovecot: auth: Debug: client in: CONT
> Mar 25 11:05:21 dovecot dovecot: auth: Debug:
> sql(u...@example.net,192.168.200.6,): query: SELECT
> NULL AS password, 'Y' as nopassword, userid AS user FROM users WHERE
> userid = 'u...@example.net'
> Mar 25 11:05:21 dovecot dovecot: auth: Debug: client in: CONT
> Mar 25 11:05:21 dovecot dovecot: auth:
> sql(u...@example.net,192.168.200.6,): Empty password
> returned without nopassword
> Mar 25 11:05:23 dovecot dovecot: auth: Debug: client passdb out:
> FAIL#0111#011user=u...@example.net
> 
> Dovecot got nopassword but does still not accept an empty password.
> 
> Christian
> 
> 
> 
> Am 31.03.2013 15:18, schrieb Timo Sirainen:
>> On 31.3.2013, at 15.47, Christian Felsing  wrote:
>> 
>>> thank you for that hint.
>>> 
>>> SELECT NULL AS password, 'Y' as nopassword, userid AS user FROM users
>>> WHERE userid = '%u'
>>> does not work, seems Dovecot 2.2rc3 ignores nopassword, so my solution is:
>> 
>> I don't understand. I remember some other mail about this as well. It works 
>> fine with my tests.. What does it log with you?
>> 
> 



Re: [Dovecot] Outlook 2013 - mounting folders with XLIST

2013-03-31 Thread Timo Sirainen
On Sun, 2013-03-31 at 17:40 +0300, Timo Sirainen wrote:
> >   
> > namespace/inbox/mailbox=hallohallo^Atnamespace/inbox/mailbox/hallohallo/auto=subscribe
> > 
> > same line occurs when querying ...userdb_import as userdb_import... in 
> > place of ...userdb_import as userdb_userdb_import…
> 
> http://hg.dovecot.org/dovecot-2.1/rev/cd919c490ea6 makes userdb_userdb_import 
> work.

Ugh. Not really. OK, latest v2.1 hg has passdb_import, userdb_import and
userdb_userdb_import and they all work. In addition, here's a list of
settings that are required for adding two new mailboxes without them
being in the configuration:

/usr/local/libexec/dovecot/imap -o 'namespace/inbox/mailbox=foo bar' -o
namespace/inbox/mailbox/foo/name=foo -o
namespace/inbox/mailbox/bar/name=bar -o
namespace/inbox/mailbox/foo/auto=create -o
namespace/inbox/mailbox/bar/auto=create -o
namespace/inbox/mailbox/foo/special_use=\\sent -o
namespace/inbox/mailbox/bar/special_use=\\drafts

That way you can easily test them. They work just the same (without -o
parameter of course) in userdb_import.



Re: [Dovecot] Outlook 2013 - mounting folders with XLIST

2013-03-31 Thread Timo Sirainen
On 25.3.2013, at 17.14, Hajo Locke  wrote:

>> Looks like it fixed the TAB problem. So I guess I need to fix 
>> userdb_userdb_import.
> 
>> Could you try if the attached patch fixes it?
> 
> i added the patch and did build my packages. sql.conf i switched back to 
> prefetch.
> Log looks like:
> 
> Mar 25 16:01:55 myhostname dovecot: auth: Debug: 
> prefetch(myusername,ip.ip.ip.ip,): success
> Mar 25 16:01:55 myhostname dovecot: auth: Debug: master out: USER 3357409281  
> myusernamehome=/home/myusername uid=1004 gid=1000
>   
> namespace/inbox/mailbox=hallohallo^Atnamespace/inbox/mailbox/hallohallo/auto=subscribe
> 
> same line occurs when querying ...userdb_import as userdb_import... in place 
> of ...userdb_import as userdb_userdb_import…

http://hg.dovecot.org/dovecot-2.1/rev/cd919c490ea6 makes userdb_userdb_import 
work.



Re: [Dovecot] Dovecot 2.2rc3 Client Cert Auth and Webmail -> auth_ssl_require_client_cert problem

2013-03-31 Thread Christian Felsing
There were log entries regarding that problem:


Mar 25 11:05:21 dovecot dovecot: auth: Debug: client in:
AUTH#0111#011PLAIN#011service=imap#011secured#011valid-client-cert#011sessi
on=J8pV8bzYIACwxigG#011cert_username=u...@example.net#011lip=192.168.200.22#011rip=192.168.200.6#011lport=993#011rport=8480
Mar 25 11:05:21 dovecot dovecot: auth: Debug: client passdb out:
CONT#0111#011
Mar 25 11:05:21 dovecot dovecot: auth: Debug: client in: CONT
Mar 25 11:05:21 dovecot dovecot: auth: Debug:
sql(u...@example.net,192.168.200.6,): query: SELECT
NULL AS password, 'Y' as nopassword, userid AS user FROM users WHERE
userid = 'u...@example.net'
Mar 25 11:05:21 dovecot dovecot: auth: Debug: client in: CONT
Mar 25 11:05:21 dovecot dovecot: auth:
sql(u...@example.net,192.168.200.6,): Empty password
returned without nopassword
Mar 25 11:05:23 dovecot dovecot: auth: Debug: client passdb out:
FAIL#0111#011user=u...@example.net

Dovecot got nopassword but does still not accept an empty password.

Christian



Am 31.03.2013 15:18, schrieb Timo Sirainen:
> On 31.3.2013, at 15.47, Christian Felsing  wrote:
> 
>> thank you for that hint.
>>
>> SELECT NULL AS password, 'Y' as nopassword, userid AS user FROM users
>> WHERE userid = '%u'
>> does not work, seems Dovecot 2.2rc3 ignores nopassword, so my solution is:
> 
> I don't understand. I remember some other mail about this as well. It works 
> fine with my tests.. What does it log with you?
> 



Re: [Dovecot] Crash while moving mail between private folder and shared folder

2013-03-31 Thread Timo Sirainen

On 27.3.2013, at 18.01, kada...@gmail.com wrote:

> Just to bump the OP message as I experience the exact same symptom with the 
> same configuration. That is: when moving an email to a virtual mailboxes, 
> dovecot crash (and sometime the email disappears from the real mailbox).

I'm guessing your error was different though. These should fix them:

http://hg.dovecot.org/dovecot-2.2/rev/12136db6e31f
http://hg.dovecot.org/dovecot-2.2/rev/9a8119326bd0

(not sure if http://hg.dovecot.org/dovecot-2.2/rev/f31d241d5371 is also 
required)

> 
> Thx
> 
> Le 23/03/13 10:03, Christian Felsing a écrit :
>> Hello,
>> 
>> while moving a mail from a private folder (inbox) to a shared folder and
>> back to private folder Dovecot 2.2rc3 crashes:
>> 
>> ---cut here---
>> Mar 23 09:57:44 dovecot dovecot: imap-login: Login:
>> user=, method=PLAIN, rip=192.168.200.6,
>> lip=192.168.200.22, mpid=19423, TLS
>> Mar 23 09:57:44 dovecot dovecot: imap(john@example.net): Panic: file
>> mail-namespace.c: line 654 (mail_namespace_find): assertion failed: (ns
>> != NULL)
>> Mar 23 09:57:44 dovecot dovecot: imap(john@example.net): Error: Raw
>> backtrace: /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot.so.0(+0x608fa)
>> [0x7f82f65b28fa] ->
>> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot.so.0(+0x6093e)
>> [0x7f82f65b293e] ->
>> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot.so.0(i_fatal+0)
>> [0x7f82f657435b] ->
>> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot-storage.so.0(mail_namespace_find_unalias+0)
>> [0x7f82f686b590] ->
>> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot-storage.so.0(mail_namespace_find_unalias+0x19)
>> [0x7f82f686b5a9] ->
>> /opt/dovecot-2.2.rc3/lib/dovecot/lib10_quota_plugin.so(+0x57bd)
>> [0x7f82f51127bd] ->
>> /opt/dovecot-2.2.rc3/lib/dovecot/lib10_quota_plugin.so(quota_test_alloc+0xcb)
>> [0x7f82f511424b] ->
>> /opt/dovecot-2.2.rc3/lib/dovecot/lib10_quota_plugin.so(quota_try_alloc+0x3d)
>> [0x7f82f511445d] ->
>> /opt/dovecot-2.2.rc3/lib/dovecot/lib10_quota_plugin.so(+0xb00b)
>> [0x7f82f511800b] ->
>> /opt/dovecot-2.2.rc3/lib/dovecot/lib01_acl_plugin.so(+0xc7cb)
>> [0x7f82f552d7cb] ->
>> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot-storage.so.0(mailbox_copy+0x5f)
>> [0x7f82f687274f] ->
>> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot-storage.so.0(mailbox_move+0x10)
>> [0x7f82f6872810] -> dovecot-01/imap() [0x40d29d] ->
>> dovecot-01/imap(command_exec+0x3c) [0x4161ec] -> dovecot-01/imap()
>> [0x415250] -> dovecot-01/imap() [0x41530a] ->
>> dovecot-01/imap(client_handle_input+0x115) [0x4155c5] ->
>> dovecot-01/imap(client_input+0x72) [0x415972] ->
>> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36)
>> [0x7f82f65c20e6] ->
>> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xd7)
>> [0x7f82f65c2f37] ->
>> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot.so.0(io_loop_run+0x28)
>> [0x7f82f65c1c28] ->
>> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot.so.0(master_service_run+0x13) 
>> [0x7f82f6579473]
>> -> dovecot-01/imap(main+0x270) [0x40b950] ->
>> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f82f61e6ead]
>> -> dovecot-01/imap() [0x40baad]
>> Mar 23 09:57:44 dovecot dovecot: imap(john@example.net): Fatal:
>> master: service(imap): child 19423 killed with signal 6 (core dumps
>> disabled)
>> Mar 23 09:57:44 dovecot dovecot: imap-login: Login:
>> user=, method=PLAIN, rip=192.168.200.6,
>> lip=192.168.200.22, mpid=19425, TLS
>> Mar 23 09:57:44 dovecot dovecot: imap(john@example.net): Panic: file
>> mail-namespace.c: line 654 (mail_namespace_find): assertion failed: (ns
>> != NULL)
>> Mar 23 09:57:44 dovecot dovecot: imap(john@example.net): Error: Raw
>> backtrace: /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot.so.0(+0x608fa)
>> [0x7f66ed9a58fa] ->
>> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot.so.0(+0x6093e)
>> [0x7f66ed9a593e] ->
>> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot.so.0(i_fatal+0)
>> [0x7f66ed96735b] ->
>> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot-storage.so.0(mail_namespace_find_unalias+0)
>> [0x7f66edc5e590] ->
>> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot-storage.so.0(mail_namespace_find_unalias+0x19)
>> [0x7f66edc5e5a9] ->
>> /opt/dovecot-2.2.rc3/lib/dovecot/lib10_quota_plugin.so(+0x57bd)
>> [0x7f66ec5057bd] ->
>> /opt/dovecot-2.2.rc3/lib/dovecot/lib10_quota_plugin.so(quota_test_alloc+0xcb)
>> [0x7f66ec50724b] ->
>> /opt/dovecot-2.2.rc3/lib/dovecot/lib10_quota_plugin.so(quota_try_alloc+0x3d)
>> [0x7f66ec50745d] ->
>> /opt/dovecot-2.2.rc3/lib/dovecot/lib10_quota_plugin.so(+0xb00b)
>> [0x7f66ec50b00b] ->
>> /opt/dovecot-2.2.rc3/lib/dovecot/lib01_acl_plugin.so(+0xc7cb)
>> [0x7f66ec9207cb] ->
>> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot-storage.so.0(mailbox_copy+0x5f)
>> [0x7f66edc6574f] ->
>> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot-storage.so.0(mailbox_move+0x10)
>> [0x7f66edc65810] -> dovecot-01/imap() [0x40d29d] ->
>> dovecot-01/imap(command_exec+0x3c) [0x4161ec] -> dovecot-01/imap()
>> [0x415250] -> dovecot-01/imap() [0x41530a] ->
>> dovecot-01/i

Re: [Dovecot] dsync mirror to convert from mbox (1.2x) to mdbox (2.2)

2013-03-31 Thread Ken A

On 3/31/2013 3:20 AM, Timo Sirainen wrote:
> On 22.3.2013, at 0.31, Ken A  wrote:
> 
>> I'm converting mail from mbox to mdbox. I've copied mail over to a
>> new system, and now I'm running dsync like this:
>> 
>> dsync -u username mirror 
>> mbox:/folder_location/username:INBOX=/inbox_location/username
>> 
>> That seemed to work, with a couple minor errors:
>> 
>>> dsync(username): Warning: Our dotlock file
>>> /mail_location/username.lock was deleted (locked 20 secs ago,
>>> touched 20 secs ago) dsync(username): Error:
>>> file_dotlock_delete() failed with mbox file
>>> /mail_location/username: No such file or directory
>> 
>> Do I also need to specify where dovecot's indexes are for the old
>> mbox files, or is this not needed for mbox conversions (when they
>> are the source)?
> 
> 
> Indexes aren't required, but looks like it takes a while to run dsync
> and you have another process that decides 20 seconds is long enough
> for a timeout and deletes the lock. Indexes would make it somewhat
> faster. Do you have mbox_dotlock_change_timeout=20? That's too low.
> 

I installed RC2.3, and am now using "dsync -R -u username backup mbox:"
and it seems to work fine now.

The only lock files would be those created by dsync, since this is all
just an rsync'd copy of mboxes that previously lived in /var/spool/mail
on another box. There are no mbox specific configurations in dovecot.

Thanks,
Ken

-- 
Ken Anderson
Pacific Internet - http://www.pacific.net


Re: [Dovecot] Crash while moving mail between private folder and shared folder

2013-03-31 Thread Timo Sirainen
On 23.3.2013, at 11.03, Christian Felsing  wrote:

> while moving a mail from a private folder (inbox) to a shared folder and
> back to private folder Dovecot 2.2rc3 crashes:
> 
> ---cut here---
> Mar 23 09:57:44 dovecot dovecot: imap-login: Login:
> user=, method=PLAIN, rip=192.168.200.6,
> lip=192.168.200.22, mpid=19423, TLS
> Mar 23 09:57:44 dovecot dovecot: imap(john@example.net): Panic: file
> mail-namespace.c: line 654 (mail_namespace_find): assertion failed: (ns
> != NULL)

Fixed: http://hg.dovecot.org/dovecot-2.2/rev/f617e2fcce66




Re: [Dovecot] Dovecot 2.2rc3 Client Cert Auth and Webmail -> auth_ssl_require_client_cert problem

2013-03-31 Thread Timo Sirainen
On 31.3.2013, at 15.47, Christian Felsing  wrote:

> thank you for that hint.
> 
> SELECT NULL AS password, 'Y' as nopassword, userid AS user FROM users
> WHERE userid = '%u'
> does not work, seems Dovecot 2.2rc3 ignores nopassword, so my solution is:

I don't understand. I remember some other mail about this as well. It works 
fine with my tests.. What does it log with you?



Re: [Dovecot] global mail_plugins

2013-03-31 Thread Timo Sirainen
On 31.3.2013, at 14.07, Andreas Meyer  wrote:

> How do I know what is specified in the global mail_plugins?
> 
> # doveconf |grep mail_plugins
> mail_plugins = 

You have nothing. Easier to test: "doveconf mail_plugins"

>  mail_plugins = " quota sieve"
>  mail_plugins = " quota imap_quota autocreate"
>  mail_plugins = " sieve"

All of these have intendation, so they are within protocol {} sections, which 
is visible if you don't use |grep directly.

> Is it just what is defined in the dovecot.conf and when there
> is nothing defined there are no global mail_plugins per default?

It's the mail_plugins setting that isn't inside any section {}.



Re: [Dovecot] Dovecot 2.2rc3 Client Cert Auth and Webmail -> auth_ssl_require_client_cert problem

2013-03-31 Thread Christian Felsing
Hi Timo,

thank you for that hint.

SELECT NULL AS password, 'Y' as nopassword, userid AS user FROM users
WHERE userid = '%u'
does not work, seems Dovecot 2.2rc3 ignores nopassword, so my solution is:

password_query = SELECT MD5('%w') AS password, userid AS user FROM users
WHERE (userid = '%u') and (('%k' = 'valid')  or ('%r' = '192.168.116.30'));

so Dovecot accepts any password provided by user. This solution works
now for users which are directily using imap or pop3 _and_ for users
which are using Horde webmail frontend backed by Dovecot. This is now a
configuration which does not need any passwords stored on server which
provides IMHO more security.

best regards
Christian


Am 31.03.2013 10:29, schrieb Timo Sirainen:
>> Is there any way to turn off client certs for specific local or remote
>> IP addresses?
> 
> In your passdb you can use %r = remote IP and %k = certificate valid to 
> figure out if the user is allowed or not. For example with SQL passdb that 
> would be possible, or checkpassword. http://wiki2.dovecot.org/Variables
> 



Re: [Dovecot] sieve-filter ignoring separator

2013-03-31 Thread Stephan Bosch

Op 3/22/2013 3:17 PM, isakrubin schreef:

Stephan Bosch-2 wrote

On 3/4/2013 9:21 PM, Isak Rubin wrote:

# dovecot --version
2.1.9

This Dovecot is very old, so is probably your Pigeonhole version. Recent
versions should work fine in this regard.

Regards,

Stephan.

Upgraded to
# dovecot --version
2.2.rc3


still same issue :/


Ok, will look at this some time next week.

Regards,

Stephan.


Re: [Dovecot] Separate log file for Sieve

2013-03-31 Thread Stephan Bosch

Op 3/31/2013 10:34 AM, Timo Sirainen schreef:

On 27.3.2013, at 16.57, Ewald Dieterich  wrote:


log_path = /var/log/dovecot/error.log
info_log_path = /var/log/dovecot/info.log
debug_log_path = /var/log/dovecot/debug.log

to log to different files.

Since I don't consider Sieve errors as severe as "real" Dovecot errors I would 
like to keep them away from the error log.

Is it possible to configure a separate log file for Sieve?

No, unless you use some syslog feature. There are some plans to reduce the 
number of errors that Sieve logs though.


Some of which is already implemented, but still pending release.

Regards,

Stephan.


[Dovecot] global mail_plugins

2013-03-31 Thread Andreas Meyer
Hello!

How do I know what is specified in the global mail_plugins?

# doveconf |grep mail_plugins
mail_plugins = 
  mail_plugins = " quota sieve"
  mail_plugins = " quota imap_quota autocreate"
  mail_plugins = " sieve"

Is it just what is defined in the dovecot.conf and when there
is nothing defined there are no global mail_plugins per default?

  Andreas


Re: [Dovecot] Cassandra support

2013-03-31 Thread Timo Sirainen
On 29.3.2013, at 11.31, Goktug YILDIRIM  wrote:

> I've read some previous threads about Cassandra support. In a message at
> April 2012 it was written that it would take a few hours after doing some
> internal stuff.
> http://dovecot.markmail.org/search/?q=cassandra#query:cassandra%20order%3Adate-backward+page:1+mid:emfxwl4fdrnrhkn7+state:results
> 
> I wonder if there have been a progress since then? Or a better alternative
> came up?


Since then I've figured out another better way to do it, basically keeping a 
local cache of mails/indexes and somewhat lazily uploading the indexes to 
storage. I've also since then figured out that only selling support doesn't 
work well enough for my Dovecot company. So we're just about ready to start 
selling object storage plugin for Dovecot, which supports all kinds of 
backends. Not Cassandra yet though, but could be added at some point. 
http://www.dovecot.fi/products/



Re: [Dovecot] Custom Headers/Flags Support in dovecot

2013-03-31 Thread Timo Sirainen
On 30.3.2013, at 3.03, Ramesh Natarajan  wrote:

> I am currently trying to evaluate if I can use dovecot to be used for my
> imap server like application. This is not an email application but a
> generic storage server that uses IMAP protocol to save and retrieve
> messages.
> 
> The messages have standard From/To/Date headers and a lot of custom headers
> for threading conversations e.t.c.  As a part of my application
> requirements I also need to support custom flags and have ability to
> set/change/retrieve them via IMAP.
> 
> 1) Does Dovecot support custom flags to be set/changed/retrieved?

Custom flags, yes. But note that they are intended to be just flags that are 
used by multiple messages, i.e. it's not a good idea to create new unique flags 
for different emails. I wouldn't be very comfortable in adding more than about 
100 different flags (per folder), although Dovecot could probably handle 
thousands (by wasting memory and CPU unnecessarily).

> 2) I understand dovecot optimizes standard RFC822 headers and flags in
> cache/index files for faster lookups and searches. Does custom headers and
> flags fall into this optimization algorithm?

Dovecot adds to cache file those headers that are requested by the client. 
Initially nothing is cached (and nothing will be if client doesn't request 
them). So only IMAP FETCH[HEADER.FIELDS (..)] or FETCH ENVELOPE or SEARCH 
HEADER xx and such will add the header to cache file. Flags are always in the 
index file.

> 3) I am planning to support multiple users and each mailbox may have upto
> 1000 folders.  Each folder may have an average of  20 MIME encoded
> messages. I am planning to use local storage with single-dbox format. Do
> you see any issues with this?

You can only change headers by creating a new message and deleting the old one. 
Sounds like IMAP ANNOTATE extension is what you're really after, but Dovecot 
doesn't support that yet. Dovecot does have IMAP METADATA plugin though, which 
allows setting key=value pairs to folders. Dovecot v2.2 will maybe have that 
natively implemented as well. It might be possible to use that as a base for 
implementing ANNOTATE extension too..



Re: [Dovecot] Postfix/Dovecot/lmtp with virtual and local users

2013-03-31 Thread Timo Sirainen
On 22.3.2013, at 11.53, David Obando  wrote:

> Timo Sirainen schrieb am 22.03.2013 09:48:
>> Maybe. Depends on your Dovecot version and passdb/userdb
>> configuration. So, doveconf -n output?
> I use version 2.1.7 from the backports repo on Debian Squeeze.
> 
> passdb {
>  args = /etc/dovecot/dovecot-sql.conf.ext
>  driver = sql
> }
> passdb {
>  driver = pam
> }

Nope, you can't currently do "user@domain" auth for sql and "user" for PAM. You 
could try using passdb checkpassword instead though, which allows you to script 
it any way you want.

> userdb {
>  args = /etc/dovecot/dovecot-sql.conf.ext
>  driver = sql
> }
> userdb {
>  driver = passwd
> }

Also userdb passwd can't do that.

I've thought of adding a generic passdb/userdb { auth_username_format } 
setting, but that doesn't exist yet. Would be easy to do though..



Re: [Dovecot] about maildirsize not updating

2013-03-31 Thread Timo Sirainen
On 29.3.2013, at 9.11, mastertoi...@netzero.net wrote:

> i am currently using roundcube my webmail and configure using dovecot quota 
> plugins. but it seems to not be updating the maildirsize when new mails 
> comes.Deleting the maildirsize will recreate the files correctly.Is there any 
> solutions to this problem with dovecot.

Either you're not using dovecot-lda to deliver the mails, or you haven't 
configured quota plugin correctly for it. dovecot -n output and logs about a 
mail delivery would help.




Re: [Dovecot] Virtual - thread refs for labeled message

2013-03-31 Thread Timo Sirainen
On 28.3.2013, at 17.35, Łukasz  wrote:

> Hi
> 
> I hope you will understand my problem.
> I want to make virtual folder called "Virtual/label1-threads". This folder 
> should have:
> 1. messages labeled fe. $label1 from INBOX
> and
> 2. messages from Sent folder inthread refs with those labeled.
> 
> message1 <-- $label1 INBOX
> |message2 <- Sent
> --|-message3 <- INBOX
> .
> 
> Can anyone help me with this?


INBOX
Sent
  inthread refs x-mailbox INBOX keyword $label1

Unfortunately currently you'll currently have trouble automatically removing 
any threads if you unset the $label1. You'd have to delete the dovecot.index* 
files so it gets rebuilt. Or alternatively use INDEX=MEMORY for the virtual 
mailbox so it never gets saved, but that's of course slower.



Re: [Dovecot] Conversion from Evolution maildir to imap

2013-03-31 Thread Timo Sirainen
On 28.3.2013, at 10.51, Paul van der Vlis  wrote:

> A customer has a thin-client server with 100+ homedirs with Evolution.
> They are using POP3 and maildir.  Now I want to switch to IMAP.
> 
> Can I use Dovecot on this maildirs, or does Evolution use another kind
> of maildir?

I'm guessing it uses Dovecot's :LAYOUT=fs directory structure.

> Does Dovecot change anything in the maildirs?
> 
> Do I loose my flags this way, like "read" ?

I'm not sure, depends on if Evolution stored the flags to maildir filenames. 
I'm guessing it doesn't, so you'd lose flags.

> Normally I am using Cyrus IMAP. What I want to do is to install Dovecot
> to have access to the mail over IMAP because Dovecot can do maildir
> (Cyrus cannot). Then I can use imapsync to sync everything to Cyrus.
> Do you think that's a good idea?

The best way in any case would be to simply configure the new IMAP account to 
Evolution and copy the messages there using Evolution itself.



Re: [Dovecot] Managesieve Automatic Homedir Creation: File not Found Error

2013-03-31 Thread Timo Sirainen
On 27.3.2013, at 18.49, linu...@netsteps.ch wrote:

> I'm running dovecot version 2.0.19 (Ubuntu 12.04) and I discovered some
> unexpected behaviour related to the Dovecot managesieve module. When
> connecting to the managesieve service from a client for the first time
> (e.g. roundcube webmail), I get an application error. This is logged by
> Dovecot:
> 
> Error: sieve-storage:
> opendir(/var/.../exampledomain/exampleuser@exampledomain//sieve) failed:
> No such file or directory.

I think there's a good chance of this being fixed in a newer version. I'd try 
with v2.1 before spending more time on it. (Although I couldn't reproduce this 
with v2.0 + latest pigeonhole either.)



Re: [Dovecot] Separate log file for Sieve

2013-03-31 Thread Timo Sirainen
On 27.3.2013, at 16.57, Ewald Dieterich  wrote:

> log_path = /var/log/dovecot/error.log
> info_log_path = /var/log/dovecot/info.log
> debug_log_path = /var/log/dovecot/debug.log
> 
> to log to different files.
> 
> Since I don't consider Sieve errors as severe as "real" Dovecot errors I 
> would like to keep them away from the error log.
> 
> Is it possible to configure a separate log file for Sieve?

No, unless you use some syslog feature. There are some plans to reduce the 
number of errors that Sieve logs though.

You could put all the LDA messages to a different log file though:

protocol lda {
  log_path = /var/log/dovecot/lda.log
}



Re: [Dovecot] Dovecot 2.2rc3 Client Cert Auth and Webmail -> auth_ssl_require_client_cert problem

2013-03-31 Thread Timo Sirainen
On 27.3.2013, at 10.49, Christian Felsing  wrote:

> I would like to set up a Dovecot based mail system which uses X.509
> Client Certificates for authentication. A webmail system based on Horde5
> should use Dovecot as backend.
..
> Unfortunately Dovecot does not support different authentication methods
> on different IP addresses or ports. This does not work:
> 
> remote 192.168.116.28/32 {
>  auth_ssl_require_client_cert = no
>  auth_ssl_username_from_cert = yes
>  disable_plaintext_auth = no
>  ssl = yes
> 
> }
> 
> Result is "doveconf: Fatal: Error in configuration file
> /opt/dovecot-2.2.rc3/etc/dovecot/conf.d/10-auth.conf line 103: Auth
> settings not supported inside local/remote blocks:
> auth_ssl_require_client_cert"

Right. Would be nice to support at some point, but not that easy to implement.

> Is there any way to turn off client certs for specific local or remote
> IP addresses?

In your passdb you can use %r = remote IP and %k = certificate valid to figure 
out if the user is allowed or not. For example with SQL passdb that would be 
possible, or checkpassword. http://wiki2.dovecot.org/Variables



Re: [Dovecot] ACL for root-level mailboxes

2013-03-31 Thread Timo Sirainen
On 26.3.2013, at 3.58, Bradley Rintoul  wrote:

> I see in the documentation where it says that if mailboxes are created at the 
> root level it will use the namespace's default ACL.  I'm using Dovecot 2.0.9 
> so 
> there is no default namespace defined, so I'm a bit handicapped here...  
> Anyway, 
> assuming I can figure out how to define a default namespace, how do I specify 
> its ACL?  Where would the file describing the access go?

I'm not sure if it works in v2.0.9 at all. But if it does, you'll simply need 
to create dovecot-acl file to the mailbox directory root (e.g. 
/path/to/Maildir/dovecot-acl).



Re: [Dovecot] dsync mirror to convert from mbox (1.2x) to mdbox (2.2)

2013-03-31 Thread Timo Sirainen
On 22.3.2013, at 0.31, Ken A  wrote:

> I'm converting mail from mbox to mdbox.
> I've copied mail over to a new system, and now I'm running dsync like this:
> 
> dsync -u username mirror
> mbox:/folder_location/username:INBOX=/inbox_location/username
> 
> That seemed to work, with a couple minor errors:
> 
>> dsync(username): Warning: Our dotlock file /mail_location/username.lock was 
>> deleted (locked 20 secs ago, touched 20 secs ago)
>> dsync(username): Error: file_dotlock_delete() failed with mbox file 
>> /mail_location/username: No such file or directory
> 
> Do I also need to specify where dovecot's indexes are for the old mbox
> files, or is this not needed for mbox conversions (when they are the
> source)?


Indexes aren't required, but looks like it takes a while to run dsync and you 
have another process that decides 20 seconds is long enough for a timeout and 
deletes the lock. Indexes would make it somewhat faster. Do you have 
mbox_dotlock_change_timeout=20? That's too low.



Re: [Dovecot] loop when I enable

2013-03-31 Thread Timo Sirainen
On 21.3.2013, at 16.55, Andreas Meyer  wrote:

> The problem occurs when I add the following to dovecot.conf:
> 
> quota_warning = storage=95%% quota-warning 95 %u
> quota_warning2 = storage=80%% quota-warning 80 %u
> quota_warning3 = -storage=100%% quota-warning below %u # user is no longer 
> over quota
> 
> The script executed looks like this:
> 
> #!/bin/sh
> PERCENT=$1
> USER=$2
> cat << EOF | /usr/libexec/dovecot/dovecot-lda -d $USER -o 
> "plugin/quota=maildir:User quota:noenforcing"

Just because quota isn't enforced, it doesn't mean that the quota warnings 
aren't executed. You need to disable quota entirely (e.g. -o plugin/quota= 
might work, although it requires a somewhat new version).