Re: dovecot-lda without starting dovecot?

[Stephan Herker]

On 11/3/2017 4:57 PM, Sami Ketola wrote:

While it might be possible to disable all the other services except master I
must ask why? How would the users be accessing their mails then?

Sami

Hello Sami,

you did not read my first post. We are talking about a multihost installation
where one host does SMTP and LDA, and the other does POP and IMAP (with
dovecot). Now we want to use dovecot-lda for the local delivery _on the SMTP
host_. So there is no need for open POP or IMAP ports and the corresponding
running services.

Again that does not answer my question why? Why do you want all the locking 
problems
and multi-access problems that come with setup like that? What is the actual 
problem that
you are trying to solve?

Sami

I think you have to run dovecot on the server receiving mails, just 
don't open imap/pop3 ports so users can't get there.  Then publish the 
other server as your imap/pop3 server.  Both servers probably should 
have the same setup for postfix and dovecot, but just kill the ports on 
the one that doesn't need it.  To run lda you're going to have to run 
dovecot on your mail relay.

Re: dovecot-lda without starting dovecot?

[Sami Ketola]

>> 
>> While it might be possible to disable all the other services except master I
>> must ask why? How would the users be accessing their mails then?
>> 
>> Sami
> 
> Hello Sami,
> 
> you did not read my first post. We are talking about a multihost installation
> where one host does SMTP and LDA, and the other does POP and IMAP (with
> dovecot). Now we want to use dovecot-lda for the local delivery _on the SMTP
> host_. So there is no need for open POP or IMAP ports and the corresponding
> running services. 

Again that does not answer my question why? Why do you want all the locking 
problems
and multi-access problems that come with setup like that? What is the actual 
problem that
you are trying to solve?

Sami

Migrating from Dovecot 1 to Dovecot 2

[Dovecot list]

Hello.
I try to migrate about 200G of mails from one server to another.
On the old i have Dovecot1 with Maildirs (without master pass etc.), on the
new one i setup dovecot2 with mdbox. I need now to migrate (partialy, not
all at once) mails from one to another.
I can't find any solution that i can use? I dont have master password, and
i want to mikgrate all mailaccont each other. Can anyone use me a working
config for this ? Best will be that migratet dont want to be downloaded by
mail client one more time.
Thanks for any help.
Best regards.

Re: iPhone/iPad IMAP connection bursts causes user+IP exceeded

[Robert Giles]

Apologies for bumping Joseph Tam's rather old thread, but I'm wondering 
if anyone has come up with a workaround/fix for this problem that iOS 
Mail.app clients (10.3.3, 11.0.3, 11.1?) continue to exhibit?


Robert



On 10/28/2016 at 03:49 PM, Joseph Tam wrote:

I frequently see this from my iPhone/iPad IMAP users:

 Oct 24 21:30:55 server dovecot: imap-login: Login: user=, ...
 [... repeated 10 times ...]
 Oct 24 21:32:54 server dovecot: imap-login: Maximum number of 
connections from user+IP exceeded (mail_max_userip_connections=12): 
user=

 Oct 24 21:32:54 server dovecot: imap(user): Logged out ...
 [... repeated 11 times ...]

These bursts of logins/max/logouts would cycling on for a few minutes.
Googling this problem seems to turn up lots of similar complaints about
iOS mail mail clients. e.g.

 https://discussions.apple.com/thread/2547839?tstart=0

iOS mail readers do not limit connections limit as other mailreaders
can.  I could increase mail_max_userip_connections, but that just moves
the goal posts.

Using the new rawlog feature in 2.2.26 (thanks Dovecot team!), I was able
to see that these connection bursts are caused by clients doing global
searches.  The rawlogs show each mailbox being SELECT'd and searched
(e.g. From header string):

 1477369968.730450 2 ID ("name" "iPad Mail" "version" "13G36" "os" 
"iOS" "os-version" "9.3.5 (13G36)")

 1477369968.781932 3 SELECT {mailbox}
 1477369968.961636 4 UID SEARCH RETURN (COUNT) 1:* NOT DELETED
 1477369969.006087 5 UID SEARCH RETURN (ALL) 1:* NOT DELETED
 1477369969.052701 6 UID SEARCH RETURN (ALL) {search-term} NOT DELETED
 1477369974.624153 7 LOGOUT

Questions:

 1) How does this affect the user?  I heard from one user that it
 makes global searches unusable because his reader just spins its
 wheel.  I'm not sure whether this is impatience or this results
 in failed searches.

 2) Is there a client-side fix (e.g. connection limiting)?
 Apple appears to be intransigent on addressing this.

 3) Will maintaining search indices (e.g. solr) help with this?
 Maybe the searches are taking too long and the connections pile
 up waiting for previous searches to finish.

Thanks,
Joseph Tam 




smime.p7s
Description: S/MIME Cryptographic Signature

Re: dovecot-lda without starting dovecot?

[Aki Tuomi]

you could try setting
protocols =
in config file to disable (most) listeners.
---Aki TuomiDovecot oy
 Original message From: Stephan von Krawczynski 
 Date: 03/11/2017  19:39  (GMT+02:00) To: Dovecot Mailing 
List  Subject: Re: dovecot-lda without starting dovecot? 
On Fri, 3 Nov 2017 19:30:22 +0200
Sami Ketola  wrote:

> > On 3 Nov 2017, at 18.23, Stephan von Krawczynski 
> > wrote: Hello Aki,
> > 
> > let me explain this a bit more. We do not intend to use only some copied
> > binary. Of course we would do a full installation of dovecot and
> > pidgeonhole, only we question if it is necessary to start the dovecot
> > init-file bringing up the dovecot imap/imap-login/pop/pop-login/auth and
> > other processes. Indeed we have virtual users.  
> 
> 
> While it might be possible to disable all the other services except master I
> must ask why? How would the users be accessing their mails then?
> 
> Sami

Hello Sami,

you did not read my first post. We are talking about a multihost installation
where one host does SMTP and LDA, and the other does POP and IMAP (with
dovecot). Now we want to use dovecot-lda for the local delivery _on the SMTP
host_. So there is no need for open POP or IMAP ports and the corresponding
running services. 

-- 
Regards,
Stephan

Re: dovecot-lda without starting dovecot?

[Stephan von Krawczynski]

On Fri, 3 Nov 2017 19:30:22 +0200
Sami Ketola  wrote:

> > On 3 Nov 2017, at 18.23, Stephan von Krawczynski 
> > wrote: Hello Aki,
> > 
> > let me explain this a bit more. We do not intend to use only some copied
> > binary. Of course we would do a full installation of dovecot and
> > pidgeonhole, only we question if it is necessary to start the dovecot
> > init-file bringing up the dovecot imap/imap-login/pop/pop-login/auth and
> > other processes. Indeed we have virtual users.  
> 
> 
> While it might be possible to disable all the other services except master I
> must ask why? How would the users be accessing their mails then?
> 
> Sami

Hello Sami,

you did not read my first post. We are talking about a multihost installation
where one host does SMTP and LDA, and the other does POP and IMAP (with
dovecot). Now we want to use dovecot-lda for the local delivery _on the SMTP
host_. So there is no need for open POP or IMAP ports and the corresponding
running services. 

-- 
Regards,
Stephan

Re: dovecot-lda without starting dovecot?

[Sami Ketola]

> On 3 Nov 2017, at 18.23, Stephan von Krawczynski  wrote:
> Hello Aki,
> 
> let me explain this a bit more. We do not intend to use only some copied
> binary. Of course we would do a full installation of dovecot and pidgeonhole,
> only we question if it is necessary to start the dovecot init-file bringing up
> the dovecot imap/imap-login/pop/pop-login/auth and other processes.
> Indeed we have virtual users.


While it might be possible to disable all the other services except master I 
must ask why?
How would the users be accessing their mails then?

Sami

Re: stats module

[Jeff Abrahamson]

On 03/11/17 17:43, Mark Moseley wrote:
>
>
> On Fri, Nov 3, 2017 at 9:35 AM, Jeff Abrahamson  > wrote:
>
> Sorry, Aki, I don't follow you.  Did I do it wrong in the file
> 91-stats
> that I shared in my original mail (attached here)?
>
> Jeff
>
>
> On 03/11/17 16:50, Aki Tuomi wrote:
> > You need to add the stats listener, by yourself.
> >
> > Aki
> >
> >> On November 3, 2017 at 5:19 PM Jeff Abrahamson  > wrote:
> >>
> >>
> >> Thanks for your suggestions, Steffen.
> >>
> >> Running doveconf -n shows no errors and also, sadly, no mention
> of the
> >> stats listener:
> >>
> >>     ╭╴ (master=)╶╮
> >>     ╰ [T] jeff@nantes-1:p27 $ doveconf -n
> >>     # 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf
> >>     # Pigeonhole version 0.4.13 (7b14904)
> >>     # OS: Linux 4.4.0-97-generic x86_64 Ubuntu 16.04.3 LTS
> >>     auth_mechanisms = plain login
> >>     auth_socket_path = /var/run/dovecot/auth-userdb
> >>     mail_location = maildir:~/Maildir
> >>     managesieve_notify_capability = mailto
> >>     managesieve_sieve_capability = fileinto reject envelope
> >>     encoded-character vacation subaddress
> comparator-i;ascii-numeric
> >>     relational regex imap4flags copy include variables body enotify
> >>     environment mailbox date index ihave duplicate mime
> foreverypart
> >>     extracttext
> >>     namespace inbox {
> >>       inbox = yes
> >>       location =
> >>       mailbox Drafts {
> >>     special_use = \Drafts
> >>       }
> >>       mailbox Junk {
> >>     special_use = \Junk
> >>       }
> >>       mailbox Sent {
> >>     special_use = \Sent
> >>       }
> >>       mailbox "Sent Messages" {
> >>     special_use = \Sent
> >>       }
> >>       mailbox Trash {
> >>     special_use = \Trash
> >>       }
> >>       prefix =
> >>     }
> >>     passdb {
> >>       driver = pam
> >>     }
> >>     plugin {
> >>       sieve = ~/.dovecot.sieve
> >>       sieve_dir = ~/sieve
> >>     }
> >>     protocols = imap sieve
> >>     service auth {
> >>       unix_listener /var/spool/postfix/private/auth {
> >>     group = postfix
> >>     mode = 0666
> >>     user = postfix
> >>       }
> >>       unix_listener /var/spool/postfix/private/dovecot-auth {
> >>     group = postfix
> >>     mode = 0660
> >>     user = postfix
> >>       }
> >>     }
> >>     service imap-login {
> >>       inet_listener imaps {
> >>     port = 993
> >>     ssl = yes
> >>       }
> >>     }
> >>     ssl_cert =  >>     ssl_cipher_list =
> >>   
>  
> EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH:+CAMELLIA256:+AES256:+CAMELLIA128:+AES128:+SSLv3:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!DSS:!RC4:!SEED:!ECDSA:CAMELLIA256-SHA:AES256-SHA:CAMELLIA128-SHA:AES128-SHA
> >>     ssl_key =  >>     ssl_protocols = !SSLv2 !SSLv3
> >>     userdb {
> >>       driver = passwd
> >>     }
> >>     protocol lda {
> >>       deliver_log_format = msgid=%m: %$
> >>       mail_plugins = sieve
> >>       postmaster_address = postmaster
> >>       quota_full_tempfail = yes
> >>       rejection_reason = Your message to <%t> was automatically
> >>     rejected:%n%r
> >>     }
> >>     protocol imap {
> >>       imap_client_workarounds = delay-newmail
> >>       mail_max_userip_connections = 20
> >>     }
> >>     protocol pop3 {
> >>       mail_max_userip_connections = 10
> >>       pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
> >>     }
> >>     ╭╴ (master=)╶╮
> >>     ╰ [T] jeff@nantes-1:p27 $
> >>
> >> Here I have a tail -f /var/log/mail.log and mail.err running in the
> >> background so we can see the results of the restart:
> >>
> >>     [T] jeff@nantes-1:conf.d $ ls -l
> >>     total 136
> >>     -rw-r--r-- 1 root root  5301 Aug 25 15:26 10-auth.conf
> >>     -rw-r--r-- 1 root root  1893 Mar 16  2016 10-director.conf
> >>     -rw-r--r-- 1 root root  2805 Mar 16  2016 10-logging.conf
> >>     -rw-r--r-- 1 root root 16172 Aug 25 15:35 10-mail.conf
> >>     -rw-r--r-- 1 root root  3547 Aug 25 15:40 10-master.conf
> >>     -rw-r--r-- 1 root root  2307 Aug 25 16:27 10-ssl.conf
> >>     -rw-r--r-- 1 root root   291 Apr 11  2017 10-tcpwrapper.conf
> >>     -rw-r--r-- 1 root root  1668 Mar 16  2016 15-lda.conf
> >>     -rw-r--r-- 1 root root  2808 Mar 16  2016 15-mailboxes.conf
> >>     -rw-r--r-- 1 root root  3295 Mar 16  2016 20-imap.conf
> >>     -rw-r--r-- 1 root root  2398 Apr 11  2017 20-managesieve.conf
> >>     -rw-r--r-- 1 root 

Re: stats module

[Mark Moseley]

On Fri, Nov 3, 2017 at 9:35 AM, Jeff Abrahamson  wrote:

> Sorry, Aki, I don't follow you.  Did I do it wrong in the file 91-stats
> that I shared in my original mail (attached here)?
>
> Jeff
>
>
> On 03/11/17 16:50, Aki Tuomi wrote:
> > You need to add the stats listener, by yourself.
> >
> > Aki
> >
> >> On November 3, 2017 at 5:19 PM Jeff Abrahamson  wrote:
> >>
> >>
> >> Thanks for your suggestions, Steffen.
> >>
> >> Running doveconf -n shows no errors and also, sadly, no mention of the
> >> stats listener:
> >>
> >> ╭╴ (master=)╶╮
> >> ╰ [T] jeff@nantes-1:p27 $ doveconf -n
> >> # 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf
> >> # Pigeonhole version 0.4.13 (7b14904)
> >> # OS: Linux 4.4.0-97-generic x86_64 Ubuntu 16.04.3 LTS
> >> auth_mechanisms = plain login
> >> auth_socket_path = /var/run/dovecot/auth-userdb
> >> mail_location = maildir:~/Maildir
> >> managesieve_notify_capability = mailto
> >> managesieve_sieve_capability = fileinto reject envelope
> >> encoded-character vacation subaddress comparator-i;ascii-numeric
> >> relational regex imap4flags copy include variables body enotify
> >> environment mailbox date index ihave duplicate mime foreverypart
> >> extracttext
> >> namespace inbox {
> >>   inbox = yes
> >>   location =
> >>   mailbox Drafts {
> >> special_use = \Drafts
> >>   }
> >>   mailbox Junk {
> >> special_use = \Junk
> >>   }
> >>   mailbox Sent {
> >> special_use = \Sent
> >>   }
> >>   mailbox "Sent Messages" {
> >> special_use = \Sent
> >>   }
> >>   mailbox Trash {
> >> special_use = \Trash
> >>   }
> >>   prefix =
> >> }
> >> passdb {
> >>   driver = pam
> >> }
> >> plugin {
> >>   sieve = ~/.dovecot.sieve
> >>   sieve_dir = ~/sieve
> >> }
> >> protocols = imap sieve
> >> service auth {
> >>   unix_listener /var/spool/postfix/private/auth {
> >> group = postfix
> >> mode = 0666
> >> user = postfix
> >>   }
> >>   unix_listener /var/spool/postfix/private/dovecot-auth {
> >> group = postfix
> >> mode = 0660
> >> user = postfix
> >>   }
> >> }
> >> service imap-login {
> >>   inet_listener imaps {
> >> port = 993
> >> ssl = yes
> >>   }
> >> }
> >> ssl_cert =  >> ssl_cipher_list =
> >> EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA384:
> EECDH+aRSA+SHA256:EECDH:+CAMELLIA256:+AES256:+CAMELLIA128:+AES128:+SSLv3:!
> aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!DSS:!RC4:!SEED:!
> ECDSA:CAMELLIA256-SHA:AES256-SHA:CAMELLIA128-SHA:AES128-SHA
> >> ssl_key =  >> ssl_protocols = !SSLv2 !SSLv3
> >> userdb {
> >>   driver = passwd
> >> }
> >> protocol lda {
> >>   deliver_log_format = msgid=%m: %$
> >>   mail_plugins = sieve
> >>   postmaster_address = postmaster
> >>   quota_full_tempfail = yes
> >>   rejection_reason = Your message to <%t> was automatically
> >> rejected:%n%r
> >> }
> >> protocol imap {
> >>   imap_client_workarounds = delay-newmail
> >>   mail_max_userip_connections = 20
> >> }
> >> protocol pop3 {
> >>   mail_max_userip_connections = 10
> >>   pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
> >> }
> >> ╭╴ (master=)╶╮
> >> ╰ [T] jeff@nantes-1:p27 $
> >>
> >> Here I have a tail -f /var/log/mail.log and mail.err running in the
> >> background so we can see the results of the restart:
> >>
> >> [T] jeff@nantes-1:conf.d $ ls -l
> >> total 136
> >> -rw-r--r-- 1 root root  5301 Aug 25 15:26 10-auth.conf
> >> -rw-r--r-- 1 root root  1893 Mar 16  2016 10-director.conf
> >> -rw-r--r-- 1 root root  2805 Mar 16  2016 10-logging.conf
> >> -rw-r--r-- 1 root root 16172 Aug 25 15:35 10-mail.conf
> >> -rw-r--r-- 1 root root  3547 Aug 25 15:40 10-master.conf
> >> -rw-r--r-- 1 root root  2307 Aug 25 16:27 10-ssl.conf
> >> -rw-r--r-- 1 root root   291 Apr 11  2017 10-tcpwrapper.conf
> >> -rw-r--r-- 1 root root  1668 Mar 16  2016 15-lda.conf
> >> -rw-r--r-- 1 root root  2808 Mar 16  2016 15-mailboxes.conf
> >> -rw-r--r-- 1 root root  3295 Mar 16  2016 20-imap.conf
> >> -rw-r--r-- 1 root root  2398 Apr 11  2017 20-managesieve.conf
> >> -rw-r--r-- 1 root root  4109 Aug 25 15:28 20-pop3.conf
> >> -rw-r--r-- 1 root root   676 Mar 16  2016 90-acl.conf
> >> -rw-r--r-- 1 root root   292 Mar 16  2016 90-plugin.conf
> >> -rw-r--r-- 1 root root  2502 Mar 16  2016 90-quota.conf
> >> -rw-r--r-- 1 root root  6822 Apr 11  2017 90-sieve.conf
> >> -rw-r--r-- 1 root root  1829 Apr 11  2017 90-sieve-extprograms.conf
> >> -rw-r--r-- 1 root root  1856 Nov  3 16:11 91-stats
> >> -rw-r--r-- 1 root root  1430 Oct 31 16:33
> 99-mail-stack-delivery.conf
> >> -rw-r--r-- 1 root root   499 Mar 16  2016
> auth-checkpassword.conf.ext
> >

Re: stats module

[Jeff Abrahamson]

Sorry, Aki, I don't follow you.  Did I do it wrong in the file 91-stats
that I shared in my original mail (attached here)?

Jeff


On 03/11/17 16:50, Aki Tuomi wrote:
> You need to add the stats listener, by yourself.
>
> Aki
>
>> On November 3, 2017 at 5:19 PM Jeff Abrahamson  wrote:
>>
>>
>> Thanks for your suggestions, Steffen.
>>
>> Running doveconf -n shows no errors and also, sadly, no mention of the
>> stats listener:
>>
>> ╭╴ (master=)╶╮
>> ╰ [T] jeff@nantes-1:p27 $ doveconf -n
>> # 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf
>> # Pigeonhole version 0.4.13 (7b14904)
>> # OS: Linux 4.4.0-97-generic x86_64 Ubuntu 16.04.3 LTS
>> auth_mechanisms = plain login
>> auth_socket_path = /var/run/dovecot/auth-userdb
>> mail_location = maildir:~/Maildir
>> managesieve_notify_capability = mailto
>> managesieve_sieve_capability = fileinto reject envelope
>> encoded-character vacation subaddress comparator-i;ascii-numeric
>> relational regex imap4flags copy include variables body enotify
>> environment mailbox date index ihave duplicate mime foreverypart
>> extracttext
>> namespace inbox {
>>   inbox = yes
>>   location =
>>   mailbox Drafts {
>>     special_use = \Drafts
>>   }
>>   mailbox Junk {
>>     special_use = \Junk
>>   }
>>   mailbox Sent {
>>     special_use = \Sent
>>   }
>>   mailbox "Sent Messages" {
>>     special_use = \Sent
>>   }
>>   mailbox Trash {
>>     special_use = \Trash
>>   }
>>   prefix =
>> }
>> passdb {
>>   driver = pam
>> }
>> plugin {
>>   sieve = ~/.dovecot.sieve
>>   sieve_dir = ~/sieve
>> }
>> protocols = imap sieve
>> service auth {
>>   unix_listener /var/spool/postfix/private/auth {
>>     group = postfix
>>     mode = 0666
>>     user = postfix
>>   }
>>   unix_listener /var/spool/postfix/private/dovecot-auth {
>>     group = postfix
>>     mode = 0660
>>     user = postfix
>>   }
>> }
>> service imap-login {
>>   inet_listener imaps {
>>     port = 993
>>     ssl = yes
>>   }
>> }
>> ssl_cert = > ssl_cipher_list =
>> 
>> EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH:+CAMELLIA256:+AES256:+CAMELLIA128:+AES128:+SSLv3:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!DSS:!RC4:!SEED:!ECDSA:CAMELLIA256-SHA:AES256-SHA:CAMELLIA128-SHA:AES128-SHA
>> ssl_key = > ssl_protocols = !SSLv2 !SSLv3
>> userdb {
>>   driver = passwd
>> }
>> protocol lda {
>>   deliver_log_format = msgid=%m: %$
>>   mail_plugins = sieve
>>   postmaster_address = postmaster
>>   quota_full_tempfail = yes
>>   rejection_reason = Your message to <%t> was automatically
>> rejected:%n%r
>> }
>> protocol imap {
>>   imap_client_workarounds = delay-newmail
>>   mail_max_userip_connections = 20
>> }
>> protocol pop3 {
>>   mail_max_userip_connections = 10
>>   pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
>> }
>> ╭╴ (master=)╶╮
>> ╰ [T] jeff@nantes-1:p27 $
>>
>> Here I have a tail -f /var/log/mail.log and mail.err running in the
>> background so we can see the results of the restart:
>>
>> [T] jeff@nantes-1:conf.d $ ls -l
>> total 136
>> -rw-r--r-- 1 root root  5301 Aug 25 15:26 10-auth.conf
>> -rw-r--r-- 1 root root  1893 Mar 16  2016 10-director.conf
>> -rw-r--r-- 1 root root  2805 Mar 16  2016 10-logging.conf
>> -rw-r--r-- 1 root root 16172 Aug 25 15:35 10-mail.conf
>> -rw-r--r-- 1 root root  3547 Aug 25 15:40 10-master.conf
>> -rw-r--r-- 1 root root  2307 Aug 25 16:27 10-ssl.conf
>> -rw-r--r-- 1 root root   291 Apr 11  2017 10-tcpwrapper.conf
>> -rw-r--r-- 1 root root  1668 Mar 16  2016 15-lda.conf
>> -rw-r--r-- 1 root root  2808 Mar 16  2016 15-mailboxes.conf
>> -rw-r--r-- 1 root root  3295 Mar 16  2016 20-imap.conf
>> -rw-r--r-- 1 root root  2398 Apr 11  2017 20-managesieve.conf
>> -rw-r--r-- 1 root root  4109 Aug 25 15:28 20-pop3.conf
>> -rw-r--r-- 1 root root   676 Mar 16  2016 90-acl.conf
>> -rw-r--r-- 1 root root   292 Mar 16  2016 90-plugin.conf
>> -rw-r--r-- 1 root root  2502 Mar 16  2016 90-quota.conf
>> -rw-r--r-- 1 root root  6822 Apr 11  2017 90-sieve.conf
>> -rw-r--r-- 1 root root  1829 Apr 11  2017 90-sieve-extprograms.conf
>> -rw-r--r-- 1 root root  1856 Nov  3 16:11 91-stats
>> -rw-r--r-- 1 root root  1430 Oct 31 16:33 99-mail-stack-delivery.conf
>> -rw-r--r-- 1 root root   499 Mar 16  2016 auth-checkpassword.conf.ext
>> -rw-r--r-- 1 root root   489 Mar 16  2016 auth-deny.conf.ext
>> -rw-r--r-- 1 root root   343 Mar 16  2016 auth-dict.conf.ext
>> -rw-r--r-- 1 root root   561 Mar 16  2016 auth-master.conf.ext
>> -rw-r--r-- 1 root root   515 Mar 16  2016 auth-passwdfile.conf.ext
>> -rw-r--r-- 1 root root   788 Mar 16  2016

Re: dovecot-lda without starting dovecot?

[Stephan von Krawczynski]

On Fri, 3 Nov 2017 17:53:47 +0200 (EET)
Aki Tuomi  wrote:

> > On November 3, 2017 at 1:50 PM Stephan von Krawczynski
> >  wrote:
> > 
> > 
> > Hello,
> > 
> > we have a setup where SMTP/LDA and POP3/IMAP are on different physical
> > hosts. They share the mail data via an external storage.
> > Now we would like to use dovecot-lda on the smtp host, so we wonder if the
> > lda binary works without starting dovecot from init. As there will be no
> > POP3/IMAP usage on this host it seems unnecessary. Nevertheless we cannot
> > judge if it is still needed for lda&sieve to work.
> > Your opinion?
> > 
> > -- 
> > Regards,
> > Stephan  
> 
> dovecot-lda does not work without dovecot unless you have physical users and
> you run the binary as target user. with virtual users it's virtually
> impossible to achieve.
> 
> Aki

Hello Aki,

let me explain this a bit more. We do not intend to use only some copied
binary. Of course we would do a full installation of dovecot and pidgeonhole,
only we question if it is necessary to start the dovecot init-file bringing up
the dovecot imap/imap-login/pop/pop-login/auth and other processes.
Indeed we have virtual users.

-- 
Regards,
Stephan

Re: dovecot-lda without starting dovecot?

[Aki Tuomi]

> On November 3, 2017 at 1:50 PM Stephan von Krawczynski  
> wrote:
> 
> 
> Hello,
> 
> we have a setup where SMTP/LDA and POP3/IMAP are on different physical hosts.
> They share the mail data via an external storage.
> Now we would like to use dovecot-lda on the smtp host, so we wonder if the
> lda binary works without starting dovecot from init. As there will be no
> POP3/IMAP usage on this host it seems unnecessary. Nevertheless we cannot
> judge if it is still needed for lda&sieve to work.
> Your opinion?
> 
> -- 
> Regards,
> Stephan

dovecot-lda does not work without dovecot unless you have physical users and 
you run the binary as target user. with virtual users it's virtually impossible 
to achieve.

Aki

Re: stats module

[Aki Tuomi]

You need to add the stats listener, by yourself.

Aki

> On November 3, 2017 at 5:19 PM Jeff Abrahamson  wrote:
> 
> 
> Thanks for your suggestions, Steffen.
> 
> Running doveconf -n shows no errors and also, sadly, no mention of the
> stats listener:
> 
> ╭╴ (master=)╶╮
> ╰ [T] jeff@nantes-1:p27 $ doveconf -n
> # 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf
> # Pigeonhole version 0.4.13 (7b14904)
> # OS: Linux 4.4.0-97-generic x86_64 Ubuntu 16.04.3 LTS
> auth_mechanisms = plain login
> auth_socket_path = /var/run/dovecot/auth-userdb
> mail_location = maildir:~/Maildir
> managesieve_notify_capability = mailto
> managesieve_sieve_capability = fileinto reject envelope
> encoded-character vacation subaddress comparator-i;ascii-numeric
> relational regex imap4flags copy include variables body enotify
> environment mailbox date index ihave duplicate mime foreverypart
> extracttext
> namespace inbox {
>   inbox = yes
>   location =
>   mailbox Drafts {
>     special_use = \Drafts
>   }
>   mailbox Junk {
>     special_use = \Junk
>   }
>   mailbox Sent {
>     special_use = \Sent
>   }
>   mailbox "Sent Messages" {
>     special_use = \Sent
>   }
>   mailbox Trash {
>     special_use = \Trash
>   }
>   prefix =
> }
> passdb {
>   driver = pam
> }
> plugin {
>   sieve = ~/.dovecot.sieve
>   sieve_dir = ~/sieve
> }
> protocols = imap sieve
> service auth {
>   unix_listener /var/spool/postfix/private/auth {
>     group = postfix
>     mode = 0666
>     user = postfix
>   }
>   unix_listener /var/spool/postfix/private/dovecot-auth {
>     group = postfix
>     mode = 0660
>     user = postfix
>   }
> }
> service imap-login {
>   inet_listener imaps {
>     port = 993
>     ssl = yes
>   }
> }
> ssl_cert =  ssl_cipher_list =
> 
> EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH:+CAMELLIA256:+AES256:+CAMELLIA128:+AES128:+SSLv3:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!DSS:!RC4:!SEED:!ECDSA:CAMELLIA256-SHA:AES256-SHA:CAMELLIA128-SHA:AES128-SHA
> ssl_key =  ssl_protocols = !SSLv2 !SSLv3
> userdb {
>   driver = passwd
> }
> protocol lda {
>   deliver_log_format = msgid=%m: %$
>   mail_plugins = sieve
>   postmaster_address = postmaster
>   quota_full_tempfail = yes
>   rejection_reason = Your message to <%t> was automatically
> rejected:%n%r
> }
> protocol imap {
>   imap_client_workarounds = delay-newmail
>   mail_max_userip_connections = 20
> }
> protocol pop3 {
>   mail_max_userip_connections = 10
>   pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
> }
> ╭╴ (master=)╶╮
> ╰ [T] jeff@nantes-1:p27 $
> 
> Here I have a tail -f /var/log/mail.log and mail.err running in the
> background so we can see the results of the restart:
> 
> [T] jeff@nantes-1:conf.d $ ls -l
> total 136
> -rw-r--r-- 1 root root  5301 Aug 25 15:26 10-auth.conf
> -rw-r--r-- 1 root root  1893 Mar 16  2016 10-director.conf
> -rw-r--r-- 1 root root  2805 Mar 16  2016 10-logging.conf
> -rw-r--r-- 1 root root 16172 Aug 25 15:35 10-mail.conf
> -rw-r--r-- 1 root root  3547 Aug 25 15:40 10-master.conf
> -rw-r--r-- 1 root root  2307 Aug 25 16:27 10-ssl.conf
> -rw-r--r-- 1 root root   291 Apr 11  2017 10-tcpwrapper.conf
> -rw-r--r-- 1 root root  1668 Mar 16  2016 15-lda.conf
> -rw-r--r-- 1 root root  2808 Mar 16  2016 15-mailboxes.conf
> -rw-r--r-- 1 root root  3295 Mar 16  2016 20-imap.conf
> -rw-r--r-- 1 root root  2398 Apr 11  2017 20-managesieve.conf
> -rw-r--r-- 1 root root  4109 Aug 25 15:28 20-pop3.conf
> -rw-r--r-- 1 root root   676 Mar 16  2016 90-acl.conf
> -rw-r--r-- 1 root root   292 Mar 16  2016 90-plugin.conf
> -rw-r--r-- 1 root root  2502 Mar 16  2016 90-quota.conf
> -rw-r--r-- 1 root root  6822 Apr 11  2017 90-sieve.conf
> -rw-r--r-- 1 root root  1829 Apr 11  2017 90-sieve-extprograms.conf
> -rw-r--r-- 1 root root  1856 Nov  3 16:11 91-stats
> -rw-r--r-- 1 root root  1430 Oct 31 16:33 99-mail-stack-delivery.conf
> -rw-r--r-- 1 root root   499 Mar 16  2016 auth-checkpassword.conf.ext
> -rw-r--r-- 1 root root   489 Mar 16  2016 auth-deny.conf.ext
> -rw-r--r-- 1 root root   343 Mar 16  2016 auth-dict.conf.ext
> -rw-r--r-- 1 root root   561 Mar 16  2016 auth-master.conf.ext
> -rw-r--r-- 1 root root   515 Mar 16  2016 auth-passwdfile.conf.ext
> -rw-r--r-- 1 root root   788 Mar 16  2016 auth-sql.conf.ext
> -rw-r--r-- 1 root root   611 Mar 16  2016 auth-static.conf.ext
> -rw-r--r-- 1 root root  2185 Mar 16  2016 auth-system.conf.ext
> -rw-r--r-- 1 root root   330 Mar 16  2016 auth-vpopmail.conf.ext
> [T] jeff@nantes-1:conf.d $ sudo service dovecot restart
>  

migrating from maildir to mdbox, preserving pop3 UIDL

[Jan-Pieter Cornet]

Hi,

I'm trying to migrate from maildir to mdbox while preserving the pop3 UIDL (and 
the imap UID).

The problem is, for maildir we use (for compatiblity with qpopper):

 pop3_uidl_format = %f

Problem is, as soon as I convert that to mdbox, then whenever a client issues 
the UIDL command via a POP connection, the connection is closed and this error 
is displayed in the log:

Error: UIDL: File name not found (pop3_uidl_format=%f not supported by storage?)

As a workaround, I enabled "pop3_reuse_xuidl = yes", and before conversion I put an X-UIDL: 
header in the mails. However, since the UIDL is the filename and the filename also contains 
S=,W=, I need to change the header in such a way that the size and 
number of lines stay the same. That's... tedious and error-prone.

Is there an easier solution, eg teaching 'doveadm sync' to preserve pop3 UIDL 
somehow?

For the record, I currently convert maildir to mdbox using:

doveadm -D backup -u user@domain 'doveadm -o 
mail=mdbox:/path/to/storage:INDEX=/path/to/fast/storage -o plugin/zlib_save=gz 
-o mail_uid=$UID -o mail_gid=$GID dsync-server'

Thanks for any help!

--
Jan-Pieter Cornet 
Systeembeheer XS4ALL Internet bv
www.xs4all.nl



signature.asc
Description: OpenPGP digital signature

Enabling notification for all users

[Delisle, John]

Hello,

I have a domain populated with ~60 users who infrequently receive email.  As a 
result, they rarely check their email, and miss important messages.  The users 
are virtual LDAP users, and each user has an attribute "backupMailAddress" that 
contains their primary high-traffic email address that they check daily.  (i.e. 
they have an address like 
j...@lowtrafficdomain.com, and a 
backupMailAddress attribute like 
j...@primaryemailaddress.com )

I'm looking to enable new-email notification for all users, and have the 
notifications go to their high-traffic email address (their backupMailAddress).

I know I can do this through individual user managesieve.sieve configurations 
and have a script that populates this with a notify rule, but this solution 
doesn't scale well, and doesn't handle new users/ departing users well.

Is the a way I could apply this globally, so that if a user has a populated 
backupMailAddress attribute, a notification is sent there?

Thanks

John Delisle | Solution Architecture | Ceridian HCM | m: 204.294.5529 
(preferred) | w: 204.975.5909
john.deli...@ceridian.com | 
www.ceridian.com
Ceridian. Makes Work Life Better(tm)



This communication is intended to be received only by the individual[s] or 
entity[s] to whom or to which it is addressed, and contains information which 
is confidential, privileged and subject to copyright.  Any unauthorized use, 
copying, review or disclosure is prohibited.  Please notify the sender 
immediately if you have received this communication in error [by calling 
collect, if necessary] so that we can arrange for its return at our expense.  
Thank you in advance for your anticipated assistance and cooperation.

Cette communication est destinée uniquement à la personne ou à la personne 
morale à qui elle est adressée. Elle contient de l’information confidentielle, 
protégée par le secret professionnel et sujette à des droits d'auteurs. Toute 
utilisation, reproduction, consultation ou divulgation non autorisées sont 
interdites. Nous vous prions d’aviser immédiatement l’expéditeur si vous avez 
reçu cette communication par erreur (en appelant à frais virés, si nécessaire), 
afin que nous puissions prendre des dispositions pour en assurer le renvoi à 
nos frais. Nous vous remercions à l’avance de votre coopération.

Re: stats module

[Jeff Abrahamson]

Thanks for your suggestions, Steffen.

Running doveconf -n shows no errors and also, sadly, no mention of the
stats listener:

╭╴ (master=)╶╮
╰ [T] jeff@nantes-1:p27 $ doveconf -n
# 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.13 (7b14904)
# OS: Linux 4.4.0-97-generic x86_64 Ubuntu 16.04.3 LTS
auth_mechanisms = plain login
auth_socket_path = /var/run/dovecot/auth-userdb
mail_location = maildir:~/Maildir
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope
encoded-character vacation subaddress comparator-i;ascii-numeric
relational regex imap4flags copy include variables body enotify
environment mailbox date index ihave duplicate mime foreverypart
extracttext
namespace inbox {
  inbox = yes
  location =
  mailbox Drafts {
    special_use = \Drafts
  }
  mailbox Junk {
    special_use = \Junk
  }
  mailbox Sent {
    special_use = \Sent
  }
  mailbox "Sent Messages" {
    special_use = \Sent
  }
  mailbox Trash {
    special_use = \Trash
  }
  prefix =
}
passdb {
  driver = pam
}
plugin {
  sieve = ~/.dovecot.sieve
  sieve_dir = ~/sieve
}
protocols = imap sieve
service auth {
  unix_listener /var/spool/postfix/private/auth {
    group = postfix
    mode = 0666
    user = postfix
  }
  unix_listener /var/spool/postfix/private/dovecot-auth {
    group = postfix
    mode = 0660
    user = postfix
  }
}
service imap-login {
  inet_listener imaps {
    port = 993
    ssl = yes
  }
}
ssl_cert =  was automatically
rejected:%n%r
}
protocol imap {
  imap_client_workarounds = delay-newmail
  mail_max_userip_connections = 20
}
protocol pop3 {
  mail_max_userip_connections = 10
  pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
}
╭╴ (master=)╶╮
╰ [T] jeff@nantes-1:p27 $

Here I have a tail -f /var/log/mail.log and mail.err running in the
background so we can see the results of the restart:

[T] jeff@nantes-1:conf.d $ ls -l
total 136
-rw-r--r-- 1 root root  5301 Aug 25 15:26 10-auth.conf
-rw-r--r-- 1 root root  1893 Mar 16  2016 10-director.conf
-rw-r--r-- 1 root root  2805 Mar 16  2016 10-logging.conf
-rw-r--r-- 1 root root 16172 Aug 25 15:35 10-mail.conf
-rw-r--r-- 1 root root  3547 Aug 25 15:40 10-master.conf
-rw-r--r-- 1 root root  2307 Aug 25 16:27 10-ssl.conf
-rw-r--r-- 1 root root   291 Apr 11  2017 10-tcpwrapper.conf
-rw-r--r-- 1 root root  1668 Mar 16  2016 15-lda.conf
-rw-r--r-- 1 root root  2808 Mar 16  2016 15-mailboxes.conf
-rw-r--r-- 1 root root  3295 Mar 16  2016 20-imap.conf
-rw-r--r-- 1 root root  2398 Apr 11  2017 20-managesieve.conf
-rw-r--r-- 1 root root  4109 Aug 25 15:28 20-pop3.conf
-rw-r--r-- 1 root root   676 Mar 16  2016 90-acl.conf
-rw-r--r-- 1 root root   292 Mar 16  2016 90-plugin.conf
-rw-r--r-- 1 root root  2502 Mar 16  2016 90-quota.conf
-rw-r--r-- 1 root root  6822 Apr 11  2017 90-sieve.conf
-rw-r--r-- 1 root root  1829 Apr 11  2017 90-sieve-extprograms.conf
-rw-r--r-- 1 root root  1856 Nov  3 16:11 91-stats
-rw-r--r-- 1 root root  1430 Oct 31 16:33 99-mail-stack-delivery.conf
-rw-r--r-- 1 root root   499 Mar 16  2016 auth-checkpassword.conf.ext
-rw-r--r-- 1 root root   489 Mar 16  2016 auth-deny.conf.ext
-rw-r--r-- 1 root root   343 Mar 16  2016 auth-dict.conf.ext
-rw-r--r-- 1 root root   561 Mar 16  2016 auth-master.conf.ext
-rw-r--r-- 1 root root   515 Mar 16  2016 auth-passwdfile.conf.ext
-rw-r--r-- 1 root root   788 Mar 16  2016 auth-sql.conf.ext
-rw-r--r-- 1 root root   611 Mar 16  2016 auth-static.conf.ext
-rw-r--r-- 1 root root  2185 Mar 16  2016 auth-system.conf.ext
-rw-r--r-- 1 root root   330 Mar 16  2016 auth-vpopmail.conf.ext
[T] jeff@nantes-1:conf.d $ sudo service dovecot restart
Nov  3 16:14:26 nantes-1 dovecot: master: Warning: Killed with
signal 15 (by pid=5845 uid=0 code=kill)
Nov  3 16:14:26 nantes-1 dovecot: imap(jeff): Server shutting down.
in=3514 out=2605
Nov  3 16:14:26 nantes-1 dovecot: master: Dovecot v2.2.22 (fe789d2)
starting up for imap, sieve (core dumps disabled)
[T] jeff@nantes-1:conf.d $

No errors there, either.

And, just to be clear that I'm not missing anything:

╭╴ (master=)╶╮
╰ [T] jeff@nantes-1:p27 $ netstat -a | grep 242
╭╴ (master=)╶╮
╰ 1,[T] jeff@nantes-1:p27 $ doveconf -n | grep stat
╭╴ (master=)╶╮
╰ 1,[T] jeff@nantes-1:p27 $

The file 91-stats contains the contents I pasted in my earlier mail. 
I'm a bit unclear what to check next.

Jeff


On 03/11/17 08:06, Steffen Kaiser wrote:
> On Thu, 2 Nov 2017, Jeff Abrahamson wrote:
>
> > In particular, nothing listens on 24242.
>
> > service stats {
> >   inet_listener

Re: LDAP Filters as defined for dovecot UserDB and passDB

[Will Merkens]

On 17-11-03 01:01 AM, Steffen Kaiser wrote:
> On Fri, 3 Nov 2017, Aki Tuomi wrote:
>
> > Check your userdb, is it using same config file?
>
> some HOWTOs explicitly want you to use two config files, because Dovecot 
> maintains two different connections for passdb and userdb queries, which is 
> faster.
> So if you have copied the config file, instead of symlinked them, you might 
> have to change two files.

Ok I checked for this.

# Authentication for LDAP users. Included from 10-auth.conf.
#
# 

passdb {
  driver = ldap

  # Path for LDAP configuration file, see example-config/dovecot-ldap.conf.ext
  args = /etc/dovecot/dovecot-ldap.conf.ext
}

# "prefetch" user database means that the passdb already provided the
# needed information and there's no need to do a separate userdb lookup.
# 
#userdb {
#  driver = prefetch
#}

userdb {
  driver = ldap
  args = /etc/dovecot/dovecot-ldap.conf.ext

  # Default fields can be used to specify defaults that LDAP may override
  #default_fields = home=/var/spool/maildir/%d/%n/Maildir
}

# If you don't have any user-specific settings, you can avoid the userdb LDAP
# lookup by using userdb static instead of userdb ldap, for example:
# 
#userdb {
  #driver = static
  #args = uid=vmail gid=vmail home=/var/vmail/%u
#}


Only one file is being called.

And in my 10-auth.conf at the bottom

# 

#!include auth-deny.conf.ext
#!include auth-master.conf.ext

#!include auth-system.conf.ext
#!include auth-sql.conf.ext
!include auth-ldap.conf.ext
#!include auth-passwdfile.conf.ext
#!include auth-checkpassword.conf.ext
#!include auth-vpopmail.conf.ext
#!include auth-static.conf.ext


ldap is the only auth system I am using.






>
> > On 02.11.2017 23:59, Will Merkens wrote:
> >> I have pretty much everything working fine.
> >>
> >> I have run into one issue with the filters that I am unsure where this a 
> >> filter is being set.
> >>
> >> Dovecot 2.2.32 (dfbe293d4)
> >>
> >> I have setup dovecot-ldap.conf.ext to control my LDAP query's
> >>
> >> My current layout used filters based on looking at the posixAccount 
> >> attributes
> >>
> >>  user_filter = (&(objectClass=posixAccount)(uid=%u))
> >>
> >> If I connect to a a ldap account that has no posixAccount attributes set I 
> >> was getting a failure in the logs
> >>
> >> Nov 02 14:15:48 mail2 dovecot[28715]: auth: Debug: 
> >> ldap(sales,192.168.123.39,): user search: 
> >> base=ou=People,dc=userful,dc=ca scope=subtree
> >> filter=(&(objectClass=posixAccount)(uid=sales)) fields=mail,user_filter
> >> Nov 02 14:15:48 mail2 dovecot[28715]: auth: Debug: 
> >> ldap(sales,192.168.123.39,): no fields returned by the 
> >> server
> >> Nov 02 14:15:48 mail2 dovecot[28715]: auth: 
> >> ldap(sales,192.168.123.39,): unknown user
> >> Nov 02 14:15:48 mail2 dovecot[28715]: auth: Error: 
> >> ldap(sales,192.168.123.39,): user not found from userdb
> >>
> >> I changed the filters thinking I needed to look at the attributes seen by 
> >> one of these accounts, I used the apache studio to find out what was 
> >> visible.
> >>
> >>> From that I saw inetOrgPerson could be used. So I changed the filters.
> >>
> >>  user_filter = (&(objectClass=inetOrgPerson)(uid=%u))
> >>  pass_filter = (&(objectClass=inetOrgPerson)(uid=%n))
> >>  iterate_filter = (objectClass=inetOrgPerson)
> >>
> >> Nov 02 14:26:44 mail2 dovecot[29047]: auth: Debug: client in: AUTH    
> >> 1    PLAIN    service=imap    secured   
> >> session=Y0GBzgVdlorAqHsn    lip=192.168.123.236
> >> Nov 02 14:26:44 mail2 dovecot[29047]: auth-worker(29066): Debug: 
> >> ldap(sa...@userful.com,192.168.123.39,): bind search:
> >> base=ou=People,dc=userful,dc=ca 
> >> filter=(&(objectClass=inetOrgPerson)(uid=sales))
> >> Nov 02 14:26:44 mail2 dovecot[29047]: auth-worker(29066): Debug: 
> >> ldap(sa...@userful.com,192.168.123.39,): result: 
> >> uid=sales; uid unused
> >> Nov 02 14:26:44 mail2 dovecot[29047]: auth-worker(29066): Debug: 
> >> ldap(sa...@userful.com,192.168.123.39,): username 
> >> changed sa...@userful.com
> >> -> sales
> >> Nov 02 14:26:44 mail2 dovecot[29047]: auth-worker(29066): Debug: 
> >> ldap(sales,192.168.123.39,): result: uid=sales
> >> Nov 02 14:26:44 mail2 dovecot[29047]: auth: Debug: 
> >> ldap(sa...@userful.com,192.168.123.39,): username 
> >> changed sa...@userful.com -> sales
> >> Nov 02 14:26:44 mail2 dovecot[29047]: auth: Debug: client passdb out: OK   
> >>  1    user=sales    original_user=sa...@userful.com
> >>
> >> so far so good but then I get
> >>
> >> Nov 02 14:26:44 mail2 dovecot[29047]: auth: Debug: master in: REQUEST  
> >>   3851550721    29049    1    519189df600c24c010b57158ac01c867 
> >>   
> >> session_pid=29073    request_auth_token
> >> Nov 02 14:26:44 mail2 dovecot[29047]: auth-worker(29066): Debug: 
> >> ldap(sales,192.168.123.39,): user search: 
> >> base=ou=People,dc=userful,dc=ca
> >> scope=subtree filter=(&(objectClass=posixAccount)(uid=sales)) 
> >> fields=mail,user_filter

Re: Bug: lmtp proxy does not quote local parts with spaces

[Stephan Bosch]

Op 3-11-2017 om 15:25 schreef David Zambonini:

On 03/11/2017 11:48, Stephan Bosch wrote:

Hi,

Sorry, we're in a bit of a v2.3 merge frenzy. Much of the LMTP code will be
replaced in v2.3, but I'll give the  older code a look as well.

This can take a while though.

Thank you very much for getting back to me, I can appreciate it can get hectic,
and I don't wish to appear ungrateful, I wholeheartedly endorse/recommend
dovecot and the company I work for does use paid for OX elsewhere. For my own
part, the platform I manage is > 300,000 mailboxes and dovecot performs
incredibly well.

I came up with some much smaller patches that accomplish the same thing in v2.2
using built-in functions and pushing the re-encoding slightly further up the
call stack - address/username being interchangeable over most of the lmtp code
makes significant changes problematic, so I thought it best not to try a rework.

Looking at gitub, though, I don't see any significant changes in behaviour as
far as the problem I'm seeing goes, which is worrying.


These changes still live in a feature branch.

Regards,

Stephan.

Re: Bug: lmtp proxy does not quote local parts with spaces

[David Zambonini]

On 03/11/2017 11:48, Stephan Bosch wrote:
> Hi,
> 
> Sorry, we're in a bit of a v2.3 merge frenzy. Much of the LMTP code will be
> replaced in v2.3, but I'll give the  older code a look as well.
> 
> This can take a while though.

Thank you very much for getting back to me, I can appreciate it can get hectic,
and I don't wish to appear ungrateful, I wholeheartedly endorse/recommend
dovecot and the company I work for does use paid for OX elsewhere. For my own
part, the platform I manage is > 300,000 mailboxes and dovecot performs
incredibly well.

I came up with some much smaller patches that accomplish the same thing in v2.2
using built-in functions and pushing the re-encoding slightly further up the
call stack - address/username being interchangeable over most of the lmtp code
makes significant changes problematic, so I thought it best not to try a rework.

Looking at gitub, though, I don't see any significant changes in behaviour as
far as the problem I'm seeing goes, which is worrying.

What I'll do is leave the patches here for reference, and pick this up again
after the v2.3 release. If you do have time for a further response, I could also
provide them as pull requests against current on github if you'd like to request
that.

1. Cut on the final instead of initial @ when splitting user/domain parts in
LMTP, this can fix some issues where localpart contains a quoted @:

dovecot-2.2.33.2-reverse-domaincut.patch

2. Fully decode local part on receipt in LMTP, and re-encode when proxying. This
fixes the issue where quoted local quotes are stripped on proxy, preventing
successful proxying, and some director hashing problems (exposes
str_append_maybe_escape in message-address.h, some logging is still
inconsistent, though, but would require a major rework):

dovecot-2.2.33.2-quoted-local-proxy.patch

-- 
David Zambonini
--- dovecot-2.2.33.2/src/lib-mail/message-address.c 2017-10-05 
18:10:44.0 +0100
+++ dovecot-2.2.33.2.quoted-local-proxy/src/lib-mail/message-address.c  
2017-11-02 12:21:57.572866755 +
@@ -34,7 +34,7 @@
 }
 
 /* quote with "" and escape all '\', '"' and "'" characters if need */
-static void str_append_maybe_escape(string_t *dest, const char *cstr, bool 
escape_dot)
+void str_append_maybe_escape(string_t *dest, const char *cstr, bool escape_dot)
 {
const char *p;
 
--- dovecot-2.2.33.2/src/lib-mail/message-address.h 2017-10-05 
18:10:44.0 +0100
+++ dovecot-2.2.33.2.quoted-local-proxy/src/lib-mail/message-address.h  
2017-11-02 13:22:45.093866755 +
@@ -39,4 +39,7 @@
  const char *address, const char **username_r,
  const char **detail_r);
 
+/* quote with "" and escape all '\', '"' and "'" characters if need */
+void str_append_maybe_escape(string_t *dest, const char *cstr, bool 
escape_dot);
+
 #endif
--- dovecot-2.2.33.2/src/lmtp/commands.c2017-10-05 18:10:44.0 
+0100
+++ dovecot-2.2.33.2.quoted-local-proxy/src/lmtp/commands.c 2017-11-02 
13:50:25.794866755 +
@@ -441,34 +441,18 @@
 
 static const char *lmtp_unescape_address(const char *name)
 {
+   struct rfc822_parser_context parser;
string_t *str;
-   const char *p;
 
if (*name != '"')
return name;
 
-   /* quoted-string local-part. drop the quotes unless there's a
-  '@' character inside or there's an error. */
+   /* decode quoted-string local-part */
str = t_str_new(128);
-   for (p = name+1; *p != '"'; p++) {
-   if (*p == '\0')
-   return name;
-   if (*p == '\\') {
-   if (p[1] == '\0') {
-   /* error */
-   return name;
-   }
-   p++;
-   }
-   if (*p == '@')
-   return name;
-   str_append_c(str, *p);
-   }
-   p++;
-   if (*p != '@' && *p != '\0')
-   return name;
+   rfc822_parser_init(&parser, (const unsigned char*)name, strlen(name), 
NULL);
+   rfc822_parse_quoted_string(&parser, str);
 
-   str_append(str, p);
+   str_append(str, (const char*)parser.data);
return str_c(str);
 }
 
--- dovecot-2.2.33.2/src/lmtp/lmtp-proxy.c  2017-10-05 18:10:44.0 
+0100
+++ dovecot-2.2.33.2.quoted-local-proxy/src/lmtp/lmtp-proxy.c   2017-11-02 
13:49:41.154866755 +
@@ -8,6 +8,7 @@
 #include "ostream.h"
 #include "str.h"
 #include "time-util.h"
+#include "message-address.h"
 #include "lmtp-client.h"
 #include "lmtp-proxy.h"
 
@@ -288,6 +289,24 @@
lmtp_proxy_try_finish(conn->proxy);
 }
 
+static char *lmtp_proxy_escape_address(pool_t pool, const char *address) {
+   const char *domain;
+   string_t *dest;
+
+   domain = strrchr(address, '@');
+   dest = str_new(pool, 128);
+
+   if (domain == NULL) {
+   str_append_maybe_escape(dest, addre

dovecot-lda without starting dovecot?

[Stephan von Krawczynski]

Hello,

we have a setup where SMTP/LDA and POP3/IMAP are on different physical hosts.
They share the mail data via an external storage.
Now we would like to use dovecot-lda on the smtp host, so we wonder if the
lda binary works without starting dovecot from init. As there will be no
POP3/IMAP usage on this host it seems unnecessary. Nevertheless we cannot
judge if it is still needed for lda&sieve to work.
Your opinion?

-- 
Regards,
Stephan

Re: Bug: lmtp proxy does not quote local parts with spaces

[Stephan Bosch]

Hi,

Sorry, we're in a bit of a v2.3 merge frenzy. Much of the LMTP code will 
be replaced in v2.3, but I'll give the  older code a look as well.


This can take a while though.

Regards,

Stephan.



Op 1-11-2017 om 18:34 schreef David Zambonini:

Hi again,

I've not heard anything further regarding this bug, so I've had a look at the 
code.

To restate the bug in a more precise way: LMTP in dovecot treats external RFC822
email addresses in the envelope recipient and internal usernames as almost
identical/interchangeable. This is incorrect and leads to issues when attempting
to use director as an LMTP proxy to proxy to recipients with quoted-local parts,
as it is issuing invalid email addresses at the LMTP protocol level (it strips
quotes from the local part and then does not add them again when proxying). It's
also causing issues with director username hashing.

I've created a "bugfix" patch to indicate what I mean, it appears to solve the
issue, although I do not think it is anywhere near a production ready change.

1. The first problem is that dovecot is not performing a full decode on the
envelope recipient address when creating the username, leading to escaped
characters left in escaped form, and is not treating it consistently, choosing
to either strip the surrounding quotes or not depending on whether or not it
contains an @. I've fixed this by changing the code in lmtp_unescape_address()
[src/lmtp/commands.c] to use rfc822_parse_quoted_string().

2. This leads to the second problem where the username becomes ambiguous if the
local-part contains an @ (regardless of whether or not the first fix is applied
or not). I've worked around this by using strrchr() instead of strchr() on the
username string to split the domain out in mail_user_hash()
[src/lib-mail/mail-user-hash.c] and message_detail_address_parse()
[src/lib-mail/message-address.c], although likely I've missed some place this
change should be made.

3. The third problem is then re-encoding the username in the envelope recipient
when proxying, which was not done at all. I've added a function
lmtp_client_rfc822_escape_address(), which is similar to
str_append_maybe_escape() to escape the address at protocol time in
lmtp_client_send_rcpts() [src/lib-smtp/lmtp-client.c], although I suspect it
should be done earlier, this is just a working proof.

The other reason I don't believe this patch is production quality is that I have
not examined any interaction between these changes and sieve's use of the
envelope recipient. I'm hoping that a developer can chip in here? (hint!)

(Apologies for top posting)

On 30/10/2017 13:18, David Zambonini wrote:

On 26/10/2017 19:33, David Zambonini wrote:

On 26/10/2017 18:38, Alexander Dalloz wrote:

Am 26.10.2017 um 12:20 schrieb David Zambonini:

There seems to be a bug with RFC822 processing in ltmp proxying that
doesn't
quote local parts that, for example, contain spaces.

Newer related RFCs are RFC 5321 and 5322.

Typo, meant to say RFC2822, which they still supercede, not that the
local-part spec has changed. :)


[ ... ]


MAIL FROM:\r\n
RCPT TO:\r\n

501 5.5.4 Invalid.parameters\r\n

That recipient address is totally invalid. It is neither just a local
part without a domain, nor a plussed address destination.

Check your setup with i.e.

RCPT TO:<"Junk E-mail"@deemzed.uk>

or

RCPT TO:<"test+Junk E-mail"@deemzed.uk>

Apologies, I was attempting to cut the config down at the time the dump
was taken. Correcting (I can provide config privately, but not share to
list), I still get:

MAIL FROM:\r\n
RCPT TO:<"deemzed.uk+Junk E-mail"@mailbox.localhost>\r\n
DATA\r\n
(etc)
.\r\n

501 5.5.4 Invalid parameters\r\n

QUIT\r\n

from director -> dovecot LMTP network dump:

I could have a look at
starting to get a fix together tomorrow with an aim to providing a pull
request, if it turns out there are no side-effects to treating
lmtp_rcpt.address like this and you'd like an example of what I mean.

My apologies for not adding your address on my initial response, Alexander - not
sure if you noticed what I replied with or not.

Nope, this isn't going to happen. I'm not familiar with the dovecot internals
but lmtp uses just the address string in the form of "full address with quotes
stripped from local part but otherwise not decoded" and nothing else throughout,
which touches on quite a bit of code. It makes it indeterminate and not always
possible to reassemble the original, it's a bit of a trainwreck.

The sanest option to me seems to me to be to store a decoded local part and
domain in addition to the detail in mail_recipient, and keeping a now properly
rfc822 encoded address in sync with it. However, this would cause a deviation
from existing behaviour for the full original user (the quotes would be seen).

I'm between a rock and a hard place here - at the very least I'd like this bug
to be officially recognised.

Re: stats module

[Steffen Kaiser]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Thu, 2 Nov 2017, Jeff Abrahamson wrote:


In particular, nothing listens on 24242.

    service stats {
  inet_listener {
    address = 127.0.0.1
    port = 24242
  }
    }


my conf looks almost identitical. This should do the trick, IMHO.

check "doveconf -n" to see, if you have a typo somewhere or any errors 
show up. Second check out the error log (there might be (re)bind 
failures). Third, make sure you have restarted Dovecot.


- -- 
Steffen Kaiser

-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQEVAwUBWfwVaXz1H7kL/d9rAQJDHgf5AY9MxGxWjEKja9w+DdiSmzSwpNuJ57Fa
kbkU8TAWQrrfJ6/uZlA7YZtFqA51HFqIO4vX+kEC2qHXJBq59x00T9CktAWAm0Cj
+48fsSiAUi8A28VW+I6GTBpDaoW1d/9aZnIg3Md+QXkD7ydWwKG3V34zXxdUE4Aw
mD8iPLCU8zjw9r0Yrwo51OA8yx+J0wRd06br50gDco09oNoQDBJueJyfhC4Dmlcr
ELulm5g2D4WOSZpSVF+wAMjgaXeC/Tf97obuGqd7aq5/F6M3Bqo60ZaBPlizUpeY
F9NSBGFCkhfSGq/mwXTWaDSveC1fG6RCW4mo85JUkhzb24kSEjEbDg==
=/yUC
-END PGP SIGNATURE-

Re: LDAP Filters as defined for dovecot UserDB and passDB

[Steffen Kaiser]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Fri, 3 Nov 2017, Aki Tuomi wrote:


Check your userdb, is it using same config file?


some HOWTOs explicitly want you to use two config files, because Dovecot 
maintains two different connections for passdb and userdb queries, which 
is faster. So if you have copied the config file, instead of symlinked 
them, you might have to change two files.



On 02.11.2017 23:59, Will Merkens wrote:

I have pretty much everything working fine.

I have run into one issue with the filters that I am unsure where this a filter 
is being set.

Dovecot 2.2.32 (dfbe293d4)

I have setup dovecot-ldap.conf.ext to control my LDAP query's

My current layout used filters based on looking at the posixAccount attributes

 user_filter = (&(objectClass=posixAccount)(uid=%u))

If I connect to a a ldap account that has no posixAccount attributes set I was 
getting a failure in the logs

Nov 02 14:15:48 mail2 dovecot[28715]: auth: Debug: 
ldap(sales,192.168.123.39,): user search: 
base=ou=People,dc=userful,dc=ca scope=subtree
filter=(&(objectClass=posixAccount)(uid=sales)) fields=mail,user_filter
Nov 02 14:15:48 mail2 dovecot[28715]: auth: Debug: 
ldap(sales,192.168.123.39,): no fields returned by the server
Nov 02 14:15:48 mail2 dovecot[28715]: auth: 
ldap(sales,192.168.123.39,): unknown user
Nov 02 14:15:48 mail2 dovecot[28715]: auth: Error: 
ldap(sales,192.168.123.39,): user not found from userdb

I changed the filters thinking I needed to look at the attributes seen by one 
of these accounts, I used the apache studio to find out what was visible.


From that I saw inetOrgPerson could be used. So I changed the filters.


 user_filter = (&(objectClass=inetOrgPerson)(uid=%u))
 pass_filter = (&(objectClass=inetOrgPerson)(uid=%n))
 iterate_filter = (objectClass=inetOrgPerson)

Nov 02 14:26:44 mail2 dovecot[29047]: auth: Debug: client in: AUTH    1 
   PLAIN    service=imap    secured   
session=Y0GBzgVdlorAqHsn    lip=192.168.123.236 
Nov 02 14:26:44 mail2 dovecot[29047]: auth-worker(29066): Debug: 
ldap(sa...@userful.com,192.168.123.39,): bind search:
base=ou=People,dc=userful,dc=ca filter=(&(objectClass=inetOrgPerson)(uid=sales))
Nov 02 14:26:44 mail2 dovecot[29047]: auth-worker(29066): Debug: 
ldap(sa...@userful.com,192.168.123.39,): result: uid=sales; 
uid unused
Nov 02 14:26:44 mail2 dovecot[29047]: auth-worker(29066): Debug: 
ldap(sa...@userful.com,192.168.123.39,): username changed 
sa...@userful.com
-> sales
Nov 02 14:26:44 mail2 dovecot[29047]: auth-worker(29066): Debug: 
ldap(sales,192.168.123.39,): result: uid=sales
Nov 02 14:26:44 mail2 dovecot[29047]: auth: Debug: 
ldap(sa...@userful.com,192.168.123.39,): username changed 
sa...@userful.com -> sales
Nov 02 14:26:44 mail2 dovecot[29047]: auth: Debug: client passdb out: OK    
1    user=sales    original_user=sa...@userful.com

so far so good but then I get

Nov 02 14:26:44 mail2 dovecot[29047]: auth: Debug: master in: REQUEST    
3851550721    29049    1    519189df600c24c010b57158ac01c867   
session_pid=29073    request_auth_token
Nov 02 14:26:44 mail2 dovecot[29047]: auth-worker(29066): Debug: 
ldap(sales,192.168.123.39,): user search: 
base=ou=People,dc=userful,dc=ca
scope=subtree filter=(&(objectClass=posixAccount)(uid=sales)) 
fields=mail,user_filter
Nov 02 14:26:44 mail2 dovecot[29047]: auth-worker(29066): Debug: 
ldap(sales,192.168.123.39,): no fields returned by the server
Nov 02 14:26:44 mail2 dovecot[29047]: auth-worker(29066): 
ldap(sales,192.168.123.39,): unknown user
Nov 02 14:26:44 mail2 dovecot[29047]: auth: Error: 
ldap(sales,192.168.123.39,): user not found from userdb

The question then is where did it get filter=(&(objectClass=posixAccount) from 
since I changed the filters to inetOrgPerson

I grep the dovecot settings directory and all My filters are in the one file. 
The file that hold the ldap settings









- -- 
Steffen Kaiser

-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQEVAwUBWfwUUnz1H7kL/d9rAQI3dggAnmkloeNU+DF1nOeKeZzEMELow9lO1DeA
vQ7eUpiXvsD/HkXdAUjF+/bd+AdUEMgZfqn8b/Wd5XhMYkcmx9w1mikYzsX8kZ/P
oHRTpfcR+pgp/FfwOeMpg64BXEUwNKcdqVeYrzVKbveWY0wJra9AifpQfFq2TXLI
1Ey4cXkqpafFcEIsjFVGzaZAtyY5+flyD/Hciyf+xP37Mpsi+pXftqzQKiH5ZV93
PVVhW7gMth6jCqIV5rvI/24rjmBqQXV+Il2/3YxLcHpvP8sL+5iis2bWSM06Lvbo
wUnMmM7mPFtRWNZfP9ClyUAIt0Qf+pu55BKbdG+bDU/i5TW6dVKs5Q==
=esxF
-END PGP SIGNATURE-