Re: [Dovecot] dsync replication errors

2013-02-16 Thread Timo Sirainen
On 17.2.2013, at 7.06, Timo Sirainen  wrote:

> On 17.2.2013, at 0.12, Michael Grimm  wrote:
> 
>> Hmm. Both jails run at distinct servers. ssh replication uses different 
>> domains, though. But, both jails are named identically "test", and both 
>> jails resolve to identical hostnames "test" if using "hostname". But, a 
>> "hostname -f" is lacking to return "test.mx1.invalid" and 
>> "test.mx2.invalid", respectively (although a "nslookup test" does). Hmm, do 
>> you think I should need to provide different hostnames in both jails? 
> 
> That's the problem most likely. I'd guess Dovecot sees both servers as having 
> "test" as the hostname and each server thinks it's the one that should be 
> doing the locking and not the other.
> 
> See if this helps: http://hg.dovecot.org/dovecot-2.2/rev/e7aabd79c9d5

Although even if it does, other parts of Dovecot still use only the hostname 
part to guarantee global uniqueness of things. So better to have unique 
hostnames.



Re: [Dovecot] dsync replication errors

2013-02-16 Thread Timo Sirainen
On 17.2.2013, at 0.12, Michael Grimm  wrote:

>>> I might add that both servers run inside FreeBSD jails (if that might make
>>> the difference to your test setup.
>> 
>> Inside jail Dovecot sees two different hostnames (same as "hostname" 
>> command)?
> 
> Hmm. Both jails run at distinct servers. ssh replication uses different 
> domains, though. But, both jails are named identically "test", and both jails 
> resolve to identical hostnames "test" if using "hostname". But, a "hostname 
> -f" is lacking to return "test.mx1.invalid" and "test.mx2.invalid", 
> respectively (although a "nslookup test" does). Hmm, do you think I should 
> need to provide different hostnames in both jails? 

That's the problem most likely. I'd guess Dovecot sees both servers as having 
"test" as the hostname and each server thinks it's the one that should be doing 
the locking and not the other.

See if this helps: http://hg.dovecot.org/dovecot-2.2/rev/e7aabd79c9d5



Re: [Dovecot] dovecot-uidlist crashing and re retriving messages by pop3 clients

2013-02-16 Thread Timo Sirainen
On 16.2.2013, at 23.41, Piotr Rotter  wrote:

> Thank you for your assistant.
> 
> Broken uidlist errors appears more than 10 time less frequently then in the 
> past and still reduce.
> 
> I have question about director and no plain text auth methods. Now I have 
> tricky query in director configuration.
> password_query = SELECT clear AS password, 'Y' as proxy, '%u' AS destuser, 
> clear AS pass FROM postfix_users WHERE email = '%u'
> 
> Is there some method to make it better.

%u as destuser isn't necessary, since it's the default.

> Master password is a bit dangerous ( man in the middle ). Is director can 
> relay no plain text authorization with out checking.

If you set NULL as password, director won't verify the password. You could even 
use passdb static. But this increases director's memory usage since users are 
verified by backends and director needs to remember all the unknown user login 
attempts (from brute force bots).

If you want non-plaintext auth:

Master password is slightly annoying, but you can set up the system so that 
director won't accept it and backends can't be connected directly.

It would also in theory be possible to use a per-user master password by using 
e.g. md5(username | master_password) as the password, so the master password 
would never be actually visible in the network (actually adding some 
non-plaintext SASL auth client support for Dovecot would basically do that).

Or the director <-> backend auth could simply have its own list of randomly 
generated passwords for users.

> I have one more question, which is not concerned subject above. Is there any 
> chance to set dovecot to be policy demon for postfix to check mailbox quota 
> before mail are checking  by (amavis, spamassassin, clamav) and before it is 
> sending by lmtp. I think that it could  reduce load.

That's something I'm planning on creating.



Re: [Dovecot] Dovecot-2.1.14 - pop3 processes always hangs forever - another follow-up

2013-02-16 Thread Timo Sirainen
On 17.2.2013, at 0.43, Axel Luttgens  wrote:

>> b) get gdb backtrace:
>> 
>> gdb -p 
>> bt full
> 
> I thus tried an eigth run, still with flocking off.
> 
> Output is in Pop3HangsGdb.txt.zip.

#21 0x7fff8f57b086 in gethostbyname ()
#22 0x00010b75571c in my_hostdomain () at hostpid.c:39

Where the code is:

hent = gethostbyname(my_hostname);

The gethostbyname() call is used to figure out the current system's domain. Why 
it would be hanging there, I don't really know. That call anyway isn't fully 
required, you could just replace it with "hent = NULL;"



Re: [Dovecot] Private message flags in shared mailboxes

2013-02-16 Thread Ben Morrow
At  8PM +0200 on 16/02/13 you (Timo Sirainen) wrote:
> 
> 1. What would be a good place to configure which flags are shared and
> which are private? [...] Another possibility would
> be in dovecot-acl file, but public mailboxes might be enabled without
> ACL plugin. 
[...]
> 
> 4. Private keywords aren't supported. I guess not much worse than
> system flags, but more code complexity. Wonder if anyone would care
> about them.

Does this mean non-owners can't set keywords, or that keywords they set
are set for everyone? I can see the latter being irritating, since some
clients (like Thunderbird) tend to set keywords without being asked to.
Denying 'w' would presumably prevent this, but that rather reduces the
point of having shared flags other than \Seen in the first place.

I think ideally one would want an extension to ACL, which supports a
per-flag (or keyword) per-mailbox per-user permission which can be
either no access, read, write shared, or write private. I don't think
this can sensibly be stuffed into the existing ACL extension (the only
reasonable place to put the flag name is as part of the user identifier,
which is confusing and crude), so this would mean a new GETFLAGACL/
SETFLAGACL/MYFLAGRIGHTS IMAP extension. Unless you think this can be
sensibly set via METADATA (I'm not familiar with that extension, so I
don't know whether using it for access control would be reasonable)?

Personally I don't think it's worth worrying about the case of shared
mailboxes without ACL.

Ben



Re: [Dovecot] Virtual Users Dovecot and Postfix

2013-02-16 Thread Ben Morrow
At 12AM +0100 on 17/02/13 you (Silvio Siefke) wrote:
> On Sat, 16 Feb 2013 03:49:37 + Ben Morrow  wrote:
> 
> > The setup given in that howto will accept mail for any user at one of
> > the domains listed in virtual_mailbox_domains, and mail for
> > nonexistent users will then be bounced by Dovecot. This is a very bad
> > idea, since you'll end up becoming a backscatter source; you should
> > set up a separate Postfix table listing the valid users at those
> > domains, and put that table in virtual_mailbox_maps. (You don't want
> > to use /etc/postfix/virtual for this, you want a separate table.)
> 
> my main.cf > http://nopaste.info/23ee74a772.html

Excerpts:

| virtual_mailbox_domains = silvio-siefke.de, silviosiefke.com
| virtual_mailbox_maps = hash:/etc/postfix/maps/mailbox
| virtual_transport = lmtp:unix:private/dovecot-lmtp
|  
| smtpd_recipient_restrictions =
| reject_unverified_recipient,

> When i understand correct the passwd file from Dovecot is enough when
> i delivered over dovecot lmtp?

Since you have 'reject_unverified_recipient', Postfix will check with
the LMTP server whether or not the recipient exists.

> Or must i set the mailboxes in the /etc/postfix/virtual too?

Did you read the text you quoted above? I specifically said you do *not*
want to use /etc/postfix/virtual for this.

What do you have in hash:/etc/postfix/maps/mailbox? If this is a
complete list of all addresses at your virtual_mailbox_domains, you can
remove reject_unverified_recipient; otherwise, you want to remove
virtual_mailbox_maps.

> Sorry for question, its long time i work with postfix and dovecot
> and in last years dovecot change much in config.

This is not really a Dovecot problem.

Ben



Re: [Dovecot] Virtual Users Dovecot and Postfix

2013-02-16 Thread Silvio Siefke
On Sat, 16 Feb 2013 03:49:37 + Ben Morrow  wrote:

> The setup given in that howto will accept mail for any user at one of
> the domains listed in virtual_mailbox_domains, and mail for
> nonexistent users will then be bounced by Dovecot. This is a very bad
> idea, since you'll end up becoming a backscatter source; you should
> set up a separate Postfix table listing the valid users at those
> domains, and put that table in virtual_mailbox_maps. (You don't want
> to use /etc/postfix/virtual for this, you want a separate table.)

my main.cf > http://nopaste.info/23ee74a772.html

When i understand correct the passwd file from Dovecot is enough when
i delivered over dovecot lmtp?

Or must i set the mailboxes in the /etc/postfix/virtual too?


Sorry for question, its long time i work with postfix and dovecot
and in last years dovecot change much in config. I want not run
a database for the mailserver, i have not so much boxes at end
running. 


Thank You for Help & Greetings
Silvio


Re: [Dovecot] dsync replication errors

2013-02-16 Thread Michael Grimm
On 16.02.2013, at 20:09, Timo Sirainen  wrote:
> On 16.2.2013, at 20.26, Michael Grimm  wrote:

>> Sorry to say, but I am still able to break replicator with v2.2.beta1
>> (35194cf0693e) under the conditions outlined below.
> 
> I wonder if locking is working correctly in your setup. Your users have home 
> directories, right?

Yes, I do have homedirs, ...

> Dovecot should be creating .dovecot-sync.lock files in there during the sync.

... and I double-checked that a .dovecot-sync.lock lockfile is being created 
during replication, and yes, it is. 

>> I might add that both servers run inside FreeBSD jails (if that might make
>> the difference to your test setup.
> 
> Inside jail Dovecot sees two different hostnames (same as "hostname" command)?

Hmm. Both jails run at distinct servers. ssh replication uses different 
domains, though. But, both jails are named identically "test", and both jails 
resolve to identical hostnames "test" if using "hostname". But, a "hostname -f" 
is lacking to return "test.mx1.invalid" and "test.mx2.invalid", respectively 
(although a "nslookup test" does). Hmm, do you think I should need to provide 
different hostnames in both jails? 

>> Good: I cannot find any "Error:" entries in both logfiles any longer.
> 
> What about Warning?

I do see only those few messages at both servers:

| dovecot: doveadm(test): Warning: fscking index file 
/.../test/storage/dovecot.map.index
| dovecot: doveadm(test): Warning: fscking index file 
/.../test/storage/dovecot.map.index
| dovecot: doveadm(test): Warning: mdbox /.../test/storage: rebuilding indexes

Please let me know what you want me to test next.

I really to appreciate your efforts and with kind regards,
Michael



Re: [Dovecot] dovecot-uidlist crashing and re retriving messages by pop3 clients

2013-02-16 Thread Piotr Rotter

Thank you for your assistant.

Broken uidlist errors appears more than 10 time less frequently then in 
the past and still reduce.


I have question about director and no plain text auth methods. Now I 
have tricky query in director configuration.
password_query = SELECT clear AS password, 'Y' as proxy, '%u' AS 
destuser, clear AS pass FROM postfix_users WHERE email = '%u'


Is there some method to make it better. Master password is a bit 
dangerous ( man in the middle ). Is director can relay no plain text 
authorization with out checking.


I have one more question, which is not concerned subject above. Is there 
any chance to set dovecot to be policy demon for postfix to check 
mailbox quota before mail are checking  by (amavis, spamassassin, 
clamav) and before it is sending by lmtp. I think that it could  reduce 
load.




Re: [Dovecot] dsync replication errors

2013-02-16 Thread Timo Sirainen
On 16.2.2013, at 20.26, Michael Grimm  wrote:

> Timo Sirainen  wrote:
> 
>> I did a bunch of dsync fixes today in hg. With the new locking
>> behavior (and other fixes) you shouldn't be able to break it anymore.
> 
> Sorry to say, but I am still able to break replicator with v2.2.beta1
> (35194cf0693e) under the conditions outlined below.

I wonder if locking is working correctly in your setup. Your users have home 
directories, right? Dovecot should be creating .dovecot-sync.lock files in 
there during the sync.

>>> This is v2.2.beta1 (78bdcb6642c7) with freshly created mailboxes "test"
>>> at both servers "mx1" and "mx2", and replicator uses ssh for remote
>>> access. Both servers run a recent postfix, use lmtp for local delivery,
>>> and "test" is a virtual user.
> 
> I might add that both servers run inside FreeBSD jails (if that might make
> the difference to your test setup.

Inside jail Dovecot sees two different hostnames (same as "hostname" command)?

> Good: I cannot find any "Error:" entries in both logfiles any longer.

What about Warning?



Re: [Dovecot] dsync replication errors

2013-02-16 Thread Oli Schacher
On Sat, 16 Feb 2013 17:20:22 +0200
Timo Sirainen  wrote:

> I did a bunch of dsync fixes today in hg. With the new locking
> behavior (and other fixes) you shouldn't be able to break it anymore.
> 

Thanks for the fixes, Timo! 

I can confirm I'm no longer able to break anything
with the tests I've mentioned so far(mass appending, simultaneous
append and delete on both mailboxes), no more errors, no more dupes.

I can also confirm the doveadm-server crash  I reported in
http://dovecot.markmail.org/thread/fb3qjnsdhtcpirg3 is now gone.

There seems to be an issue left when expunging a large amount of
messages from the Trash. I managed to get it twice so far by expunging
~3k messages. I'll try to create a reproducible test script for this
scenario. I can currently only provide my "clicking around" log output.
Version is current hg, e63d1cf19ec7.

First time it happened:
Feb 16 18:49:48 doco2 dovecot: imap(user1): Warning: Maildir 
/mailstore/user1/maildir/.Trash: Expunged message reappeared, giving a new UID 
(old uid=1221, file=1361035457.M728795P6220.doco1,S=2476,W=2555:2,Sa)
Feb 16 18:49:48 doco2 dovecot: imap(user1): Warning: Maildir 
/mailstore/user1/maildir/.Trash: Expunged message reappeared, giving a new UID 
(old uid=1222, file=1361035458.M501466P6220.doco1,S=2477,W=2556:2,Sa)
Feb 16 18:49:48 doco2 dovecot: imap(user1): Warning: Maildir 
/mailstore/user1/maildir/.Trash: Expunged message reappeared, giving a new UID 
(old uid=1223, file=1361035458.M988177P6220.doco1,S=2520,W=2599:2,Sa)
Feb 16 18:49:48 doco2 dovecot: imap(user1): Warning: Maildir 
/mailstore/user1/maildir/.Trash: Expunged message reappeared, giving a new UID 
(old uid=1224, file=1361035459.M254031P6220.doco1,S=2483,W=2562:2,Sa)
Feb 16 18:49:49 doco2 dovecot: imap(user1): Warning: Maildir 
/mailstore/user1/maildir/.Trash: Expunged message reappeared, giving a new UID 
(old uid=1225, file=1361035459.M431911P6220.doco1,S=2490,W=2569:2,Sa)
Feb 16 18:49:49 doco2 dovecot: imap(user1): Warning: Maildir 
/mailstore/user1/maildir/.Trash: Expunged message reappeared, giving a new UID 
(old uid=1226, file=1361035459.M959244P6220.doco1,S=2482,W=2561:2,Sa)
Feb 16 18:50:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Error: 
Couldn't lock /mailstore/user1/.dovecot-sync.lock: Interrupted system call
Feb 16 18:50:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Error: 
dsync(local): Received unexpected input S != H
Feb 16 18:50:14 doco2 dovecot: dsync-local(user1): Error: 
read(vmail@192.168.23.61) failed: EOF
Feb 16 18:50:14 doco2 dovecot: dsync-local(user1): Error: Remote command 
returned error 75
Feb 16 18:50:44 doco2 dovecot: doveadm: Error: dsync-remote(user1): Error: 
Couldn't lock /mailstore/user1/.dovecot-sync.lock: Interrupted system call
Feb 16 18:50:44 doco2 dovecot: doveadm: Error: dsync-remote(user1): Error: 
dsync(local): Received unexpected input N != H
Feb 16 18:50:44 doco2 dovecot: dsync-local(user1): Error: 
read(vmail@192.168.23.61) failed: EOF
Feb 16 18:50:44 doco2 dovecot: dsync-local(user1): Error: Remote command 
returned error 75


2nd time: (no "reappeared" messages this time)
Feb 16 19:08:13 doco2 dovecot: imap-login: Login: user=, method=PLAIN, 
rip=192.168.23.130, lip=192.168.23.62, mpid=4794, session=
Feb 16 19:08:44 doco2 dovecot: doveadm: Error: dsync-remote(user1): Error: 
Couldn't lock /mailstore/user1/.dovecot-sync.lock: Interrupted system call
Feb 16 19:08:44 doco2 dovecot: doveadm: Error: dsync-remote(user1): Error: 
dsync(local): Received unexpected input S != H
Feb 16 19:08:44 doco2 dovecot: dsync-local(user1): Error: 
read(vmail@192.168.23.61) failed: EOF
Feb 16 19:08:44 doco2 dovecot: dsync-local(user1): Error: Remote command 
returned error 75


A while later on the other server:
Feb 16 19:13:08 doco1 dovecot: doveadm: Error: dsync-remote(user1): Panic: file 
mail-transaction-log-view.c: line 72 (mail_transaction_log_view_set): assertion 
failed: (min_file_seq <= max_file_seq)
Feb 16 19:13:08 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Raw 
backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x5dc2a) [0x7f305f325c2a] -> 
/usr/lib64/dovecot/libdovecot.so.0(default_fatal_handler+0x32) [0x7f305f325d12] 
-> /usr/lib64/dovecot/libdovecot.so.0(+0x1f80a) [0x7f305f2e780a] -> 
/usr/lib64/dovecot/libdovecot-storage.so.0(mail_transaction_log_view_set+0x580) 
[0x7f305f64e3f0] -> /usr/bin/doveadm() [0x43786b] -> 
/usr/bin/doveadm(dsync_transaction_log_scan_init+0x8c) [0x43791c] -> 
/usr/bin/doveadm(dsync_brain_sync_mailbox_open+0x5e) [0x42724e] -> 
/usr/bin/doveadm(dsync_brain_slave_recv_mailbox+0x123) [0x427c63] -> 
/usr/bin/doveadm(dsync_brain_run+0x178) [0x425ff8] -> /usr/bin/doveadm() 
[0x4265d1] -> /usr/bin/doveadm() [0x4357f0] -> 
/usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f305f334bd6] -> 
/usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0xa7) [0x7f305f335c67] 
-> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f305f334b78] -
 > /usr/bin/doveadm() [0x424294] -> 

Re: [Dovecot] dsync replication errors

2013-02-16 Thread Michael Grimm
Timo Sirainen  wrote:

> I did a bunch of dsync fixes today in hg. With the new locking
> behavior (and other fixes) you shouldn't be able to break it anymore.

Sorry to say, but I am still able to break replicator with v2.2.beta1
(35194cf0693e) under the conditions outlined below.

> On 2013-02-01 Michael Grimm wrote:
> 
>> This is v2.2.beta1 (78bdcb6642c7) with freshly created mailboxes "test"
>> at both servers "mx1" and "mx2", and replicator uses ssh for remote
>> access. Both servers run a recent postfix, use lmtp for local delivery,
>> and "test" is a virtual user.

I might add that both servers run inside FreeBSD jails (if that might make
the difference to your test setup.

>> All tests are run with vanilla mailboxes, after restarting dovecot, and
>> without imap connections by MUA:

This time I did even restart both service jails before every test. And, I
did use both Mail.app and roundcube as MUA to check the results (if Mail.app
might have screwed INBOX ...)

>> 1) Simultaneous mailbomb approach: run both scripts simultaneously, and
>>   you'll end up with numerous duplicates in mailboxes "test". Very often
>>   you'll find multiples.

Still a lot of duplicates and multiples. Those numbers are not reproducable,
240 (best case) up to 340 (worst case) instead of 200 messages (after 10 
tests).
 
Here is one logfile example of a triplicated mail injected at mx1:

logfile at mx1:
| Feb 16 19:03:12  mx1 postfix/pickup[33958]: 3Z7fMh1PYMz5Ng: uid=0 
from=
| Feb 16 19:03:12  mx1 postfix/cleanup[34320]: 3Z7fMh1PYMz5Ng: 
message-id=<3Z7fMh1PYMz5Ng@test.mx1.invalid>
| Feb 16 19:03:12  mx1 postfix/qmgr[33959]: 3Z7fMh1PYMz5Ng: 
from=, size=310, nrcpt=1 (queue active)
| Feb 16 19:03:12  mx1 dovecot: lmtp(34456, test): copy from : 
box=INBOX, uid=12, msgid=<3Z7fMh1PYMz5Ng@test.mx1.invalid>, size=544, 
from=root@mx1.invalid (admin), flags=()
| Feb 16 19:03:12  mx1 dovecot: lmtp(34456, test): 
nVlIDeDJH1GYhgAAag1aAg: sieve: msgid=<3Z7fMh1PYMz5Ng@test.mx1.invalid>: stored 
mail into mailbox 'INBOX'
| Feb 16 19:03:12  mx1 postfix/lmtp[34453]: 3Z7fMh1PYMz5Ng: 
to=, orig_to=, 
relay=test.mx1.invalid[private/dovecot-lmtp], delay=0.29, delays=0.08/0/0/0.21, 
dsn=2.0.0, status=sent (250 2.0.0  nVlIDeDJH1GYhgAAag1aAg 
Saved)
| Feb 16 19:03:12  mx1 postfix/qmgr[33959]: 3Z7fMh1PYMz5Ng: removed
| Feb 16 19:03:13  mx1 dovecot: dsync-local(test): copy from INBOX: 
box=INBOX, uid=42, msgid=<3Z7fMh1PYMz5Ng@test.mx1.invalid>, size=544, 
from=root@mx1.invalid (admin), flags=()
| Feb 16 19:03:13  mx1 dovecot: dsync-local(test): expunge: 
box=INBOX, uid=12, msgid=<3Z7fMh1PYMz5Ng@test.mx1.invalid>, size=544, 
from=root@mx1.invalid (admin), flags=(\Recent)
| Feb 16 19:03:16  mx1 dovecot: dsync-local(test): copy from INBOX: 
box=INBOX, uid=164, msgid=<3Z7fMh1PYMz5Ng@test.mx1.invalid>, size=544, 
from=root@mx1.invalid (admin), flags=()
| Feb 16 19:03:16  mx1 dovecot: dsync-local(test): copy from INBOX: 
box=INBOX, uid=263, msgid=<3Z7fMh1PYMz5Ng@test.mx1.invalid>, size=544, 
from=root@mx1.invalid (admin), flags=()
| Feb 16 19:03:16  mx1 dovecot: dsync-local(test): expunge: 
box=INBOX, uid=118, msgid=<3Z7fMh1PYMz5Ng@test.mx1.invalid>, size=544, 
from=root@mx1.invalid (admin), flags=(\Recent)
| Feb 16 19:03:16  mx1 dovecot: dsync-local(test): expunge: 
box=INBOX, uid=42, msgid=<3Z7fMh1PYMz5Ng@test.mx1.invalid>, size=544, 
from=root@mx1.invalid (admin), flags=(\Recent)

after reading those three messages at mx1:
| Feb 16 19:04:22  mx1 dovecot: imap(test) hQjfUNvVPwBd3Cqw: 
flag_change: box=INBOX, uid=372, msgid=<3Z7fMh1PYMz5Ng@test.mx1.invalid>, 
size=544, from=root@mx1.invalid (admin), flags=(\Seen \Recent)
| Feb 16 19:05:40  mx1 dovecot: imap(test) hQjfUNvVPwBd3Cqw: 
flag_change: box=INBOX, uid=263, msgid=<3Z7fMh1PYMz5Ng@test.mx1.invalid>, 
size=544, from=root@mx1.invalid (admin), flags=(\Seen \Recent)
| Feb 16 19:05:41  mx1 dovecot: imap(test) hQjfUNvVPwBd3Cqw: 
flag_change: box=INBOX, uid=164, msgid=<3Z7fMh1PYMz5Ng@test.mx1.invalid>, 
size=544, from=root@mx1.invalid (admin), flags=(\Seen \Recent)

logfile at mx2:
| Feb 16 19:03:13  mx2 dovecot: dsync-local(test): save: box=INBOX, 
uid=50, msgid=<3Z7fMh1PYMz5Ng@test.mx1.invalid>, size=544, 
from=root@mx1.invalid (admin), flags=()
| Feb 16 19:03:17  mx2 dovecot: dsync-local(test): copy from INBOX: 
box=INBOX, uid=372, msgid=<3Z7fMh1PYMz5Ng@test.mx1.invalid>, size=544, 
from=root@mx1.invalid (admin), flags=()

>> 2) Mailbomb approach: run one script at one server only, and all mails
>>   will become perfectly well synchronised.

Same results here.

>> 3) Modify both scripts to "( $INDEX % 1 == 0 )" to add a second waiting
>>   between every mail injection, and run them simultaneously at both
>>   servers, and you'll end up with significantly less duplicates and no
>>   more multiples.

Same results here.

Good: I cannot find any "Error:" entries in both logfiles any longer.

Regards,
Michael



[Dovecot] Private message flags in shared mailboxes

2013-02-16 Thread Timo Sirainen
I'm wondering a bit how these should work. Comments welcome.

In v2.2 the recommended way to enable private flags for shared mailboxes is to 
create private indexes with :INDEXPVT=. This creates 
dovecot.index.pvt[.log] files which only contain message UIDs and flags. The 
indexes are updated somewhat lazily by updating them whenever user has the 
shared mailbox opened and it gets synced.

1. What would be a good place to configure which flags are shared and which are 
private? Currently it forces \Seen flag to be private and others not. With 
Maildir I used to have the idea of configuring these in dovecot-shared file, 
but that's a rather annoying extra file. Then again it could be cached into 
dovecot.index. Another possibility would be in dovecot-acl file, but public 
mailboxes might be enabled without ACL plugin. Mailbox attributes (for URLAUTH 
and for future METADATA) also exist in v2.2, but they require configuring a 
dict for them, which isn't very efficient either to read every time a mailbox 
is accessed. Mailbox's admin user should be able to do this somehow via IMAP 
protocol also. METADATA would pretty much be required for that I guess.

2. Private flags aren't currently inherited from the primary (shared) flags. 
With user-shared mailboxes this is good, because the owner doesn't have private 
indexes for the mailboxes, so when owner reads a mail it gets a \Seen flag in 
the primary index. If that mailbox was shared to someone else, the \Seen flag 
shouldn't be set. But this also means that for truly shared (public) mailboxes 
that have no owner you can't set initial flags with Sieve's addflag command 
(e.g. \Seen flag to high-scored mails in Spam). I guess it could be possible to 
change this so that the inheritance is done for public namespaces but not for 
shared namespaces, but is that really a good idea either?..

3. Currently COPY/APPEND ignore any private flags (which also means dsync won't 
set them always). Fixing this would require remembering these flags, committing 
the save to primary index, syncing the private index to get the new mails added 
there, then updating their flags (or doing it directly at sync stage with some 
kludging code). To avoid race conditions the private index would need to be 
locked before committing the save. I guess this is doable..

4. Private keywords aren't supported. I guess not much worse than system flags, 
but more code complexity. Wonder if anyone would care about them.



Re: [Dovecot] Defer emails until the user is ready to deal with them (snooze button / bring-forward file / bring-up file for emails)

2013-02-16 Thread Daniel Parthey
Hi Chris,

Chris Laif wrote:
> After reading a message, I would like to decide that this very specific
> message should disappear from my mailbox for some time (I'am not able to
> handle/answer this email today, so it should reappear tomorrow/next
> week/...).

Well, I simply move some mail into a folder called "Todo" and
as soon as I've got time, handle all mails in this folder.

You could even write a script, which marks the messages in the Todo
folder via IMAP as "new" or "unread" after a specified time,
but this is rather a client-side issue and should not be handled
by Dovecot.

Regards
Daniel
-- 
https://plus.google.com/103021802792276734820


Re: [Dovecot] doveadm -D quota get -u user

2013-02-16 Thread Robert Schetterer
Am 16.02.2013 17:27, schrieb Daniel Parthey:
> Hi Robert,
> 
> Robert Schetterer wrote:
>> dovecot --version
>> 2.1.12 (abcc9959b757)
>>
>> just running into problems with
>>
>> doveadm -D quota get -u user
>> doveadm(root): Fatal: Unknown command 'quota', but plugin quota exists.
>> Try to set mail_plugins=quota
>>
>> should mail_plugins=quota really be set ?
> 
> Sure, at least in the global setting,
> might be different for protocol imap or protocol lmtp.
> 
> http://wiki2.dovecot.org/Tools/Doveadm/Quota
> 
> The quota get and quota recalc commands are only available when the global
> mail_plugins setting contains the quota plugin.

guilty, thx , i overread this
it works now

> 
> Regards
> Daniel
> 



Best Regards
MfG Robert Schetterer

-- 
[*] sys4 AG

http://sys4.de, +49 (89) 30 90 46 64
Franziskanerstraße 15, 81669 München

Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer
Aufsichtsratsvorsitzender: Joerg Heidrich


Re: [Dovecot] Defer emails until the user is ready to deal with them (snooze button / bring-forward file / bring-up file for emails)

2013-02-16 Thread Chris Laif
On Sat, Feb 16, 2013 at 5:43 PM, Timo Sirainen  wrote:
>> As far as I've learned this is not possible with IMAP:
>> - storing the 'do-not-show-until-time' within a custom header-field
>> requires down- and uploading of the whole email and therefore causes
>> problems especially with large emails
>> - IMAP keywords are limited are limited in quantity and can not be
>> compared with datetime functions
>
> http://tools.ietf.org/html/rfc5257 could be used for this. Of course,
> Dovecot doesn't support it, and it's not easy to implement efficiently.
>
> Probably not too difficult to implement inefficiently with v2.2's
> mailbox_attribute_* API. Just create /annotations// mailbox
> attributes and delete them when the mails get expunged..
>

Thanks Timo, at least one person understands me ;-) Unfortunately,
both solutions require programming skills far better than I have. So I
start enjoying my weekend doing other things ... thank you for your
great work on dovecot!

Chris


Re: [Dovecot] dovecot index errors since 2.1.12

2013-02-16 Thread Morten Stevens

On 29.01.2013 16:57, Timo Sirainen wrote:

On Tue, 2013-01-22 at 16:38 +0200, Timo Sirainen wrote:

On Mon, 2013-01-14 at 16:16 +0100, Daniel Parthey wrote:

Jan 12 16:48:35 10.129.3.233 dovecot: mailbox: mail:
imap(us...@example.org): : Error: Cached message
size larger than expected (5194 > 4399)


I wonder if this patch causes your lmtp/lda processes to start
assert-crashing (which at least with lmtp causes MTA to retry, with lda
might reject mail depending on config):
http://hg.dovecot.org/dovecot-2.1/rev/2b76d357a56a

Since the bug appears to be in the mail delivery part, it would be much
easier to debug&fix it by looking at what goes wrong there.


Fixed: http://hg.dovecot.org/dovecot-2.1/rev/0b0399f1b6aa


Hi Timo,

Thank you. I can confirm the bug has been fixed since 2.1.14.

Best regards,

Morten


Re: [Dovecot] "Renaming not supported across conflicting directory permissions"

2013-02-16 Thread Timo Sirainen
On Wed, 2013-02-13 at 13:04 -0500, Richard Platel wrote:
> Is it possible to disable this check in 2.1.7? (without a patch and 
> re-compile?)

Nope. I know there are some fixes related to that check in a newer v2.1.
If it doesn't work with v2.1.15 either the way you want, let me know
what exactly is the problem.




Re: [Dovecot] dovecot-uidlist crashing and re retriving messages by pop3 clients

2013-02-16 Thread Timo Sirainen
On Wed, 2013-02-13 at 23:43 +0100, Piotr Rotter wrote:
> I configure director temporary in some tricky way because I have to 
> serve md5 and ntlm authentication method and I do not want reconfigure 
> pop/imap servers.
> 
> Can you confirm I done all ok becouse I have that issue:
> 
>   # doveadm director map
> doveadm(root): Error: User listing returned failure
> doveadm(root): Error: user listing failed
> user 

All is working. If you want "doveadm director map" command to work you
need to either:

a) Make "doveadm user '*'" command work by providing appropriate userdb
configuration or

b) use doveadm director map -f userdb parameter where userdb is a file
that lists all the usernames you have.

There's no need to use doveadm director map command at all though.




Re: [Dovecot] proxy imaps

2013-02-16 Thread Timo Sirainen
On Thu, 2013-02-14 at 10:48 +0100, Antonio Casado Rodriguez wrote:
> Hi all,
> 
> I had a imaps dovecot server working normal. I wanted to isolate the 
> Internet connection by placing a proxy imap(Intranet connections still 
> connect directly to the imap server).
> Yesterday I set up oneproxy imaps dovecot and thereafter, imap dovecot 
> logs on errors:
> 
> imap-login: Error: OpenSSL malloc() failed. You may need to increase 
> login_process_size
> 
> Need dovecot more memory with imap proxy connections than direct 
> connections?

Yes, SSL needs more memory. The error message is obsolete though, it
should have said to increase service imap-login { vsz_limit }. Fixed in
hg.




Re: [Dovecot] Segfault with doveadm-server

2013-02-16 Thread Timo Sirainen
On Thu, 2013-02-14 at 13:16 +0400, Evgeny Basov wrote:
> Hello.
> 
> With dovecot ver. from 20130211 I have a segfault on doveadm when sync.
> There are messages from server:
> 
> 2013-02-14T12:42:35.508031+04:00 imap1 kernel: [870542.927083]
> doveadm-server[9483]: segfault at 4 ip 0074b228bc24 sp
> 03f30b8e61b0 error 4 in doveadm-server[74b226b000+3a000]
> 
> I cannot catch process doveadm-server by strace because it starts with
> unknown PID on demand.

strace isn't useful for debugging crashes, gdb backtrace is. You should
be able to get a core dump from doveadm-server (I've definitely been
debugging those today many times). http://dovecot.org/bugreport.html

Anyway, the crash is probably fixed by today's changes.




Re: [Dovecot] Defer emails until the user is ready to deal with them (snooze button / bring-forward file / bring-up file for emails)

2013-02-16 Thread Reindl Harald


Am 16.02.2013 17:50, schrieb Chris Laif:
>> if you do not want to see new mails do not check for
>> them - no need to mangle messages
>>
>> switch TB in the offline-mode or clsoe the mail-client
>> based on WHAT will wahtever you think you need to implement
>> dec ide "if the user is ready"
>>
> 
> Sorry, but that's not what I meant. After reading a message, I would
> like to decide that this very specific message should disappear from
> my mailbox for some time (I'am not able to handle/answer this email
> today, so it should reappear tomorrow/next week/...)

and this is why you would touch the message-headers?
what the hell - IMAP supports folders and subfolders

under my inbox are 30 subfolders and i get between
300 and 500 messages per day with a usual response
time of 5 minutes as you can see becuase some of them
are important and some of them are archived

for reminders -> thunderbird lightning

as said - this is a social and not a technical problem





signature.asc
Description: OpenPGP digital signature


Re: [Dovecot] Defer emails until the user is ready to deal with them (snooze button / bring-forward file / bring-up file for emails)

2013-02-16 Thread Chris Laif
> if you do not want to see new mails do not check for
> them - no need to mangle messages
>
> switch TB in the offline-mode or clsoe the mail-client
> based on WHAT will wahtever you think you need to implement
> dec ide "if the user is ready"
>

Sorry, but that's not what I meant. After reading a message, I would
like to decide that this very specific message should disappear from
my mailbox for some time (I'am not able to handle/answer this email
today, so it should reappear tomorrow/next week/...).

I agree that this transforms email into a kind of todo/task-list. But
many people I know are using email-accounts this way. Gmail and
mailboxapp.com (and probably more) offer similar features, just google
for 'snooze email'.

Very likely it is not necessary to modifiy Thunderbird (or any other
email-client) for this. You just have to set up a filter folder
'do-not-show-until-time'>now().

Chris


Re: [Dovecot] Dovecot-2.1.14 - pop3 processes always hangs forever - another follow-up

2013-02-16 Thread Timo Sirainen
On Fri, 2013-02-15 at 17:32 +0100, Axel Luttgens wrote:

>  a pop connection hangs forever after the pass command
> 
>   $ telnet 127.0.0.1 110
>   Trying 127.0.0.1...
>   Connected to localhost.
>   Escape character is '^]'.
>   +OK Dovecot ready.
>   user user2
>   +OK
>   pass pass2
> 
> and the log shows (here, when the user connects for the first time)
> 
>   dovecot[2997]: pop3-login: Login: user=, method=PLAIN, 
> rip=127.0.0.1, lip=127.0.0.1, mpid=3113, secured, session=
>   dovecot[2997]: pop3(user2): Debug: Effective uid=13, gid=13, 
> home=/_Data/Mailstores/13
>   dovecot[2997]: pop3(user2): Debug: Home dir not found: 
> /_Data/Mailstores/13
>   dovecot[2997]: pop3(user2): Debug: fs: 
> root=/_Data/Mailstores/13/mboxes, index=, control=, 
> inbox=/_Data/Mailstores/13/mboxes/inbox, alt=
>   dovecot[2997]: pop3(user2): Debug: Namespace : 
> /_Data/Mailstores/13/mboxes doesn't exist yet, using default permissions
>   dovecot[2997]: pop3(user2): Debug: Namespace : Using permissions from 
> /_Data/Mailstores/13/mboxes: mode=0700 gid=-1
>   dovecot[2997]: pop3(user2): Debug: Namespace : 
> /_Data/Mailstores/13/mboxes/inbox doesn't exist yet, using default 
> permissions
>   dovecot[2997]: pop3(user2): Debug: Namespace : Using permissions from 
> /_Data/Mailstores/13/mboxes: mode=0700 gid=-1
>   dovecot[2997]: pop3(user2): Debug: Namespace : 
> /_Data/Mailstores/13/mboxes/inbox doesn't exist yet, using default 
> permissions
>   dovecot[2997]: pop3(user2): Debug: Namespace : Using permissions from 
> /_Data/Mailstores/13/mboxes: mode=0700 gid=-1
> 
> With exactly the same config, imap connections are fully functional.
> 
> Does someone have an explanation?

a) dtruss the pop3 process to see which syscall hangs. Maybe this works:

service pop3 {
  executable = /usr/bin/dtruss /usr/local/libexec/dovecot/pop3
}

It logs to Dovecot's error log.

b) get gdb backtrace:

gdb -p 
bt full




Re: [Dovecot] Defer emails until the user is ready to deal with them (snooze button / bring-forward file / bring-up file for emails)

2013-02-16 Thread Timo Sirainen
On Sat, 2013-02-16 at 12:44 +0100, Chris Laif wrote:
> Hi, I would like to add a feature to my email-client (Thunderbird)
> which stops emails from showing up until a specific time (e.g. having
> buttons for: show again this evening, tomorrow, next week, next month
> ...).
> 
> As far as I've learned this is not possible with IMAP:
> - storing the 'do-not-show-until-time' within a custom header-field
> requires down- and uploading of the whole email and therefore causes
> problems especially with large emails
> - IMAP keywords are limited are limited in quantity and can not be
> compared with datetime functions

http://tools.ietf.org/html/rfc5257 could be used for this. Of course,
Dovecot doesn't support it, and it's not easy to implement efficiently.

Probably not too difficult to implement inefficiently with v2.2's
mailbox_attribute_* API. Just create /annotations// mailbox
attributes and delete them when the mails get expunged..




Re: [Dovecot] dovecot-2.2: dsync to imapc not working

2013-02-16 Thread Timo Sirainen
On Mon, 2013-02-11 at 12:07 +0400, Evgeny Basov wrote:
> 11.02.2013 03:15, Timo Sirainen пишет:
> > Should work better now in latest hg. 
> 
> # dovecot --version
> 20130211 (ed5fa76f0dc7+)
> 
> Thanks. Work better but not fine. Not all emails was syncrornized:

I did several dsync fixes today. See if it happens to work now? "doveadm
backup -R imapc:" works at least in my quick tests.




Re: [Dovecot] Dsync converting dbox->Maildir

2013-02-16 Thread Timo Sirainen
On Wed, 2013-02-13 at 12:28 +0400, Anes Mukhametov wrote:

> I've changed my mailbox format from dbox to Maildir. The problem is maildir
> message filename extension has no S=/W= field after migration.
> Using dovecot 2.1.15.

How did you change it? dsync is the only tool that can properly do it,
and it definitely adds the S/W fields.

Anyway, Dovecot can handle maildir files without S/W files anyway. But
if you really want to add them, use
http://dovecot.org/tools/maildir-size-fix.pl with -a parameter. It adds
only S though, no W support yet.




Re: [Dovecot] doveadm -D quota get -u user

2013-02-16 Thread Daniel Parthey
Hi Robert,

Robert Schetterer wrote:
> dovecot --version
> 2.1.12 (abcc9959b757)
>
> just running into problems with
> 
> doveadm -D quota get -u user
> doveadm(root): Fatal: Unknown command 'quota', but plugin quota exists.
> Try to set mail_plugins=quota
> 
> should mail_plugins=quota really be set ?

Sure, at least in the global setting,
might be different for protocol imap or protocol lmtp.

http://wiki2.dovecot.org/Tools/Doveadm/Quota

The quota get and quota recalc commands are only available when the global
mail_plugins setting contains the quota plugin.

Regards
Daniel
-- 
https://plus.google.com/103021802792276734820


[Dovecot] doveadm -D quota get -u user

2013-02-16 Thread Robert Schetterer
Hi Timo,

just running into problems with

doveadm -D quota get -u user


doveadm(root): Debug: Loading modules from directory:
/usr/lib/dovecot/modules
doveadm(root): Debug: Module loaded:
/usr/lib/dovecot/modules/lib01_acl_plugin.so
doveadm(root): Debug: Module loaded:
/usr/lib/dovecot/modules/lib15_notify_plugin.so
doveadm(root): Debug: Module loaded:
/usr/lib/dovecot/modules/lib20_autocreate_plugin.so
doveadm(root): Debug: Module loaded:
/usr/lib/dovecot/modules/lib20_expire_plugin.so
doveadm(root): Debug: Module loaded:
/usr/lib/dovecot/modules/lib20_fts_plugin.so
doveadm(root): Debug: Module loaded:
/usr/lib/dovecot/modules/lib20_listescape_plugin.so
doveadm(root): Debug: Module loaded:
/usr/lib/dovecot/modules/lib20_mail_log_plugin.so
doveadm(root): Debug: Module loaded:
/usr/lib/dovecot/modules/lib20_virtual_plugin.so
doveadm(root): Debug: Module loaded:
/usr/lib/dovecot/modules/lib20_zlib_plugin.so
doveadm(root): Debug: Module loaded:
/usr/lib/dovecot/modules/lib21_fts_lucene_plugin.so
doveadm(root): Debug: Loading modules from directory:
/usr/lib/dovecot/modules/doveadm
doveadm(root): Debug: Module loaded:
/usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so
doveadm(root): Debug: Module loaded:
/usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so
doveadm(root): Debug: Skipping module doveadm_quota_plugin, because
dlopen() failed:
/usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so:
undefined symbol: quota_user_module (this is usually intentional, so
just ignore this message)
doveadm(root): Debug: Module loaded:
/usr/lib/dovecot/modules/doveadm/lib10_doveadm_zlib_plugin.so
doveadm(root): Debug: Module loaded:
/usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_lucene_plugin.so
doveadm(root): Debug: Module loaded:
/usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so
doveadm(root): Fatal: Unknown command 'quota', but plugin quota exists.
Try to set mail_plugins=quota

should mail_plugins=quota really be set ?


dovecot --version
2.1.12 (abcc9959b757)

Best Regards
MfG Robert Schetterer

-- 
[*] sys4 AG

http://sys4.de, +49 (89) 30 90 46 64
Franziskanerstraße 15, 81669 München

Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer
Aufsichtsratsvorsitzender: Joerg Heidrich


Re: [Dovecot] dsync replication errors

2013-02-16 Thread Timo Sirainen
I did a bunch of dsync fixes today in hg. With the new locking behavior
(and other fixes) you shouldn't be able to break it anymore.

On Fri, 2013-02-01 at 21:53 +0100, Michael Grimm wrote:
> [Sorry Oli for my previous mail to your address, only. Resent here]
> 
> Oli Schacher  wrote:
> 
> > There still seems to be a problem when changes to both mailboxes at
> > the same time are involved
> 
> I can confirm your observation, although triggered by a different test
> scenario, similar to the one I did use with 2.1 replicator before
> (http://www.dovecot.org/list/dovecot/2012-March/064354.html).
> 
> This is v2.2.beta1 (78bdcb6642c7) with freshly created mailboxes "test"
> at both servers "mx1" and "mx2", and replicator uses ssh for remote
> access. Both servers run a recent postfix, use lmtp for local delivery,
> and "test" is a virtual user.
> 
> Test script to produce local testmails of equal size at mx1:
> | #!/bin/csh
> | set INDEX= 101
> | set endINDEX = 200
> | while ( $INDEX <= $endINDEX )
> |echo $INDEX
> |echo "test" | mail -s $INDEX test@mx1
> |if ( $INDEX % 1000 == 0 ) then
> |   sleep 1
> |endif
> |@ INDEX = $INDEX + 1
> |end
> |exit 0
> 
> Test script to produce testmails of equal size at mx2:
> | #!/bin/csh
> | set INDEX= 1101
> | set endINDEX = 1200
> | while ( $INDEX <= $endINDEX )
> |echo $INDEX
> |echo "test" | mail -s $INDEX test@mx2
> |if ( $INDEX % 1000 == 0 ) then
> |   sleep 1
> |endif
> |@ INDEX = $INDEX + 1
> |end
> |exit 0
> 
> All tests are run with vanilla mailboxes, after restarting dovecot, and
> without imap connections by MUA:
> 
> 1) Simultaneous mailbomb approach: run both scripts simultaneously, and
>   you'll end up with numerous duplicates in mailboxes "test". Very often
>   you'll find multiples.
> 
> 2) Mailbomb approach: run one script at one server only, and all mails
>   will become perfectly well synchronised.
> 
> 3) Mofify both scripts to "( $INDEX % 1 == 0 )" to add a second waiting
>   between every mail injection, and run them simultaneously at both
>   servers, and you'll end up with significantly less duplicates and no
>   more multiples.
> 
> > Feb  1 07:12:52 doco1 dovecot: dsync-local(user1): Error: Mailbox INBOX: 
> > Remote didn't send mail GUID=7a30ff22af5b0b510f0c960042f4 (UID=211)
> 
> > Feb  1 07:12:54 doco2 dovecot: dsync-local(user1): Error: Importing mailbox 
> > INBOX failed
> 
> > Feb  1 07:13:24 doco2 dovecot: dsync-local(user1): Error: Remote command 
> > process isn't dying, killing it
> 
> I do see those error messages as well, and in addition numerous of those:
> 
> | dovecot: dsync-local(test): Error: Mailbox INBOX: Unexpected GUID mismatch 
> for UID=7153: 82c5df0a4ffa0b5141e36a0d5a02 != 
> 29cc9f284ffa0b5141c236abecbd
> 
> | doveadm: Error: dsync-remote(test): Error: Mailbox INBOX: Unexpected GUID 
> mismatch for UID=7153: 82c5df0a4ffa0b5141e36a0d5a02 != 
> 29cc9f284ffa0b5141c236abecbd
> 
> | dovecot: lmtp(49752, test): Error: Corrupted index cache file 
> /.../test/mailboxes/INBOX/dbox-Mails/dovecot.index.cache: File too small
> 
> | Feb  1 18:35:16   mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: 
> Error: Mailbox INBOX: Corrupted index, uidvalidity=0
> | Feb  1 18:35:16  mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: 
> Warning: fscking index file /.../test/storage/dovecot.map.index
> | Feb  1 18:35:16   mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: 
> Error: Mailbox INBOX: Corrupted index, uidvalidity=0
> | Feb  1 18:35:16   mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: 
> Error: mdbox /.../test/mailboxes/INBOX/dbox-Mails: Storage keeps breaking
> | Feb  1 18:35:16   mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: 
> Error: Mailbox INBOX: Corrupted index, uidvalidity=0
> | Feb  1 18:35:16  mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: 
> Warning: fscking index file /.../test/storage/dovecot.map.index
> | Feb  1 18:35:16  mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: 
> Warning: mdbox /.../test/storage: rebuilding indexes
> | Feb  1 18:35:17  mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: 
> Warning: fscking index file /.../test/storage/dovecot.map.index
> | Feb  1 18:35:17  mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: 
> Warning: fscking index file /.../test/storage/dovecot.map.index
> | Feb  1 18:35:17  mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: 
> Warning: mdbox /.../test/storage: rebuilding indexes
> | Feb  1 18:35:17  mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: 
> Warning: fscking index file /.../test/storage/dovecot.map.index
> | Feb  1 18:35:17  mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: 
> Warning: fscking index file /.../test/storage/dovecot.map.index
> | Feb  1 18:35:17  mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: 
> Warning: mdbox /.../test/storage: rebuilding indexes
> | Feb  1 18:35:18  mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: 
> Warning: fscking index file /.../test/storage/dovecot.map.index
> | Feb  1 18:35:18  mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: 
> Warning: fscking index file /.../t

Re: [Dovecot] Broken physical size caching in Dovecot 2.1.10

2013-02-16 Thread Timo Sirainen
On Wed, 2013-02-13 at 11:49 +, Rob Redpath wrote:
> > Dovecot's zlib plugin requires that the maildir filenames
> > have ,S=uncompressed size. Otherwise you get those errors.
> > http://dovecot.org/tools/maildir-size-check.sh can be used to fix the
> > filenames.
> >
> Thanks for this. The problem I'm encountering is that Dovecot renames 
> files, and does so to their size on disk rather than their uncompressed 
> size, undoing any corrective action that's been taken. 

Yeah. It's a bit difficult to change the automated fixing code to work
properly with compressed files without some very ugly kludges.

BTW. http://dovecot.org/tools/maildir-size-fix.pl has been updated to
work with compressed files also, making maildir-size-check.sh obsolete.




Re: [Dovecot] Defer emails until the user is ready to deal with them (snooze button / bring-forward file / bring-up file for emails)

2013-02-16 Thread Robert Schetterer
Am 16.02.2013 13:46, schrieb Chris Laif:
> On Sat, Feb 16, 2013 at 12:49 PM, Reindl Harald  
> wrote:
>> you said "I would like to add a feature to my email-client" so why
>> do you need to add headers to them? what has the server to do with
>> this all and why would you touch the mails for this?
>>
> 
> People are accessing their (IMAP-)inboxes from multiple
> computers/devices these days, so all information has to be stored on
> the server.
> 
> Chris
> 

perhaps

http://tools.ietf.org/html/rfc3864

may helpfull



Best Regards
MfG Robert Schetterer

-- 
[*] sys4 AG

http://sys4.de, +49 (89) 30 90 46 64
Franziskanerstraße 15, 81669 München

Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer
Aufsichtsratsvorsitzender: Joerg Heidrich


Re: [Dovecot] Defer emails until the user is ready to deal with them (snooze button / bring-forward file / bring-up file for emails)

2013-02-16 Thread Reindl Harald


Am 16.02.2013 13:46, schrieb Chris Laif:
> On Sat, Feb 16, 2013 at 12:49 PM, Reindl Harald  
> wrote:
>> you said "I would like to add a feature to my email-client" so why
>> do you need to add headers to them? what has the server to do with
>> this all and why would you touch the mails for this?
>>
> 
> People are accessing their (IMAP-)inboxes from multiple
> computers/devices these days, so all information has to be stored on
> the server.

sorry but this all is nonsense and the context "i want to
implement a feature in TB but multiple devices" it makes
no sense

if you do not want to see new mails do not check for
them - no need to mangle messages

switch TB in the offline-mode or clsoe the mail-client
based on WHAT will wahtever you think you need to implement
dec ide "if the user is ready"





signature.asc
Description: OpenPGP digital signature


Re: [Dovecot] Defer emails until the user is ready to deal with them (snooze button / bring-forward file / bring-up file for emails)

2013-02-16 Thread Chris Laif
On Sat, Feb 16, 2013 at 12:49 PM, Reindl Harald  wrote:
> you said "I would like to add a feature to my email-client" so why
> do you need to add headers to them? what has the server to do with
> this all and why would you touch the mails for this?
>

People are accessing their (IMAP-)inboxes from multiple
computers/devices these days, so all information has to be stored on
the server.

Chris


Re: [Dovecot] Defer emails until the user is ready to deal with them (snooze button / bring-forward file / bring-up file for emails)

2013-02-16 Thread Robert Schetterer
Am 16.02.2013 12:44, schrieb Chris Laif:
> Hi, I would like to add a feature to my email-client (Thunderbird)
> which stops emails from showing up until a specific time (e.g. having
> buttons for: show again this evening, tomorrow, next week, next month
> ...).

yeah ,there are lot of nice new ideas with mail these days

> 
> As far as I've learned this is not possible with IMAP:
> - storing the 'do-not-show-until-time' within a custom header-field
> requires down- and uploading of the whole email and therefore causes
> problems especially with large emails
> - IMAP keywords are limited are limited in quantity and can not be
> compared with datetime functions
> 
> Any suggestions will be greatly appreciated!

however some server coders will may include new features,
that does not solve the client problem,
the only existing free major cross os imap client recent is thunderbird,
but i dont see mozilla willing to push more development in it.
After all ,there is lot of "old" bleeding edge stuff for left implement
in Thunderbird, before including new ideas.


> 
> Chris
> 



Best Regards
MfG Robert Schetterer

-- 
[*] sys4 AG

http://sys4.de, +49 (89) 30 90 46 64
Franziskanerstraße 15, 81669 München

Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer
Aufsichtsratsvorsitzender: Joerg Heidrich


Re: [Dovecot] Defer emails until the user is ready to deal with them (snooze button / bring-forward file / bring-up file for emails)

2013-02-16 Thread Reindl Harald


Am 16.02.2013 12:44, schrieb Chris Laif:
> Hi, I would like to add a feature to my email-client (Thunderbird)
> which stops emails from showing up until a specific time (e.g. having
> buttons for: show again this evening, tomorrow, next week, next month
> ...).
> 
> As far as I've learned this is not possible with IMAP:
> - storing the 'do-not-show-until-time' within a custom header-field
> requires down- and uploading of the whole email and therefore causes
> problems especially with large emails

sounds more like a social problem then a technical
touching the e-mails is a very very bad idea because you may break them
what does it help DOWNLOAD the mails, hide them and touch them

you said "I would like to add a feature to my email-client" so why
do you need to add headers to them? what has the server to do with
this all and why would you touch the mails for this?

if the user is not ready to deal with mails -> close TB



signature.asc
Description: OpenPGP digital signature


[Dovecot] Defer emails until the user is ready to deal with them (snooze button / bring-forward file / bring-up file for emails)

2013-02-16 Thread Chris Laif
Hi, I would like to add a feature to my email-client (Thunderbird)
which stops emails from showing up until a specific time (e.g. having
buttons for: show again this evening, tomorrow, next week, next month
...).

As far as I've learned this is not possible with IMAP:
- storing the 'do-not-show-until-time' within a custom header-field
requires down- and uploading of the whole email and therefore causes
problems especially with large emails
- IMAP keywords are limited are limited in quantity and can not be
compared with datetime functions

Any suggestions will be greatly appreciated!

Chris


Re: [Dovecot] Virtual Users Dovecot and Postfix

2013-02-16 Thread Ben Morrow
At  8AM +0100 on 16/02/13 you (Thomas Leuxner) wrote:
> * Ben Morrow  2013.02.16 04:49:
> 
> > > http://wiki2.dovecot.org/HowTo/VirtualUserFlatFilesPostfix
> > 
> > The setup given in that howto will accept mail for any user at one of
> > the domains listed in virtual_mailbox_domains, and mail for nonexistent
> > users will then be bounced by Dovecot. This is a very bad idea, since
> > you'll end up becoming a backscatter source; you should set up a
> > separate Postfix table listing the valid users at those domains, and put
> > that table in virtual_mailbox_maps. (You don't want to use
> > /etc/postfix/virtual for this, you want a separate table.)
> 
> It will verify the recipients before accepting the mail:
> 
> reject_unverified_recipient
> http://www.postfix.org/postconf.5.html#reject_unverified_recipient

Hang on; the first listing in the howto uses that option, the second
doesn't. I was looking at the second, as generally the cleaner. Hmmph.

In any case, I referred to that option, but I don't really think it's a
good idea. It seems like unnecessary overhead just to avoid building an
extra map.

> I don't think the overhead of duplicating user checks on both ends is
> justified until this process is automated.

The SMTP server has to make user checks of some sort.
reject_unverified_recipient makes those checks more expensive, not less,
since some of the time it will have to call out to the LMTP server to
verify an address.

Building a Postfix map from a passwd-format file is trivial. It can
easily be incorporated into whatever script is used for adding and
removing users already, or into /etc/postfix/Makefile or whatever is
used for rebuilding the Postfix maps.

> Above will instruct the
> SMTP Server to verify the recipient address before accepting mail.
> This is what you want and it works dynamic out of the box. A better
> reason to use separate aliasing on the Postfix side is to rewrite
> addresses for the Dovecot Mailboxes. If you'd want to have a mailbox
> with several addresses in Dovecot you'd need to create dummy users for
> this while you can just alias them in Postfix.

This would be done with virtual aliases in Postfix, which are separate
from the virtual mailboxes which end up delivered to Dovecot. (In fact,
under many circumstances, it's more convenient to use local_transport
for delivery to Dovecot.)

Ben