Re: about header address parsing

[TACHIBANA Masashi]

Hi,

Thank you for your reply.

--
Tachibana

- Original Message -
> On Tue, 1 Sep, 2020 at 09:59, Timo Sirainen  wrote:
> > On 1. Sep 2020, at 6.24, TACHIBANA Masashi  
> > wrote:
> >> 
> >> Hi,
> >> 
> >> Is this expected or not?
> >> 
> >> From: us...@fuga.example.com 
> >> To: us...@hoge.example.com 
> >> ↓
> >> a uid fetch 43055 (envelope)
> >> * 1860 FETCH (UID 43055 ENVELOPE ("Thu, 30 Jul 2020 13:52:59 +0900" 
> >> "test1" ((NIL NIL "user1" "fuga.example.com")) ((NIL NIL "user1" 
> >> "fuga.example.com")) ((NIL NIL "user1" "fuga.example.com")) ((NIL 
> >> NIL "user2" "hoge.example.com")) NIL NIL NIL 
> >> ""))
> > 
> > This is an invalid email address, so it's neither correct nor 
> > incorrect to have this output. But this reminded me that I was going 
> > to discuss about this with other IETF people. Lets see what others 
> > think: 
> > https://mailarchive.ietf.org/arch/msg/extra/sqRTdsV_DGBhHu2ghdCDFo_pM8Q/
> 
> While it is an invalid email address, in the exact same vein as 
>  
> Dovecot's approach is unhelpful here, and means MUAs must download 
> complete headers rather than reply on envelope address structures. In 
> fact, unlike in the linked case this example this is actually a 
> security vulnerability: http://mailspolit.com/
> 
> As a MUA maintainer, I'd really like to see Dovecot take a more 
> proactive approach to sending useful values in envelope address 
> structure, so we don't have to download headers all the time.
> 
> 
> >> From: "us...@fuga.example.com" 
> >> To: "us...@hoge.example.com" 
> >> ↓
> >> a uid fetch 43056 (envelope)
> >> * 1861 FETCH (UID 43056 ENVELOPE ("Thu, 30 Jul 2020 13:53:59 +0900" 
> >> "test1" (("us...@fuga.example.com" NIL "user1" "example.com")) 
> >> (("us...@fuga.example.com" NIL "user1" "example.com")) 
> >> (("us...@fuga.example.com" NIL "user1" "example.com")) 
> >> (("us...@hoge.example.com" NIL "user2" "example.com")) NIL NIL NIL 
> >> ""))
> > 
> > This is a valid email address, and ENVELOPE reply is correct.
> 
> Agreed.
> 
> //Mike
> 
> -- 
> Michael Gratton.
> 
> 
> 
> 
--
TACHIBANA Masashi  QUALITIA CO., LTD.
mailto:tachib...@qualitia.co.jp

株式会社クオリティア
https://www.qualitia.co.jp/

Dsync replication - delayed replication (Sync lock)

[Daniel Botting]

Hi,

*Our setup:*

Two Debian 10 machines that are setup to replicate mail between them, we 
have round robin DNS setup so a user can connect to either server.


*What should happen:*

Mail is delivered to either server and replicated across straight away 
to their mailbox on the other server so it does not matter which one 
they are connected to they will receive it fairly soon after delivery.


*What actually happens:*

In some instances the user will experience a delayed receipt of messages 
if they are not connected to the server that the message is initially 
delivered to, sometimes the delay is 5/10 minutes, we had a recent 
support ticket submitted where it was over an hour.


Error message seen in mail.err:

Sep  1 10:16:15  dovecot: 
dsync-local(): Error: Couldn't lock 
/path/to/mailbox/.dovecot-sync.lock: 
fcntl(/path/to/mailbox/.dovecot-sync.lock, write-lock, F_SETLKW) locking 
failed: Timed out after 30 seconds (WRITE lock held by pid 3697)


Process 3697 is dovecot/doveadm-server.

*Doveconf -n output:*

# 2.3.10.1 (a3d0e1171): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.10 (67bf5bd7)
# OS: Linux 4.19.0-10-amd64 x86_64 Debian 10.5
# Hostname: 
auth_verbose = yes
default_vsz_limit = 0
doveadm_password = # hidden, use -P to show it
first_valid_gid = 8
first_valid_uid = 8
last_valid_gid = 8
last_valid_uid = 8
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
mail_gid = 8
mail_location = maildir:~/Maildir
mail_plugins = " notify replication"
mail_uid = 8
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope 
encoded-character vacation subaddress comparator-i;ascii-numeric 
relational regex im
ap4flags copy include variables body enotify environment mailbox date 
index ihave duplicate mime foreverypart extracttext editheader imapfla

gs
namespace inbox {
  inbox = yes
  location =
  mailbox Drafts {
    special_use = \Drafts
  }
  mailbox Junk {
    special_use = \Junk
  }
  mailbox Sent {
    special_use = \Sent
 }
  mailbox "Sent Messages" {
    special_use = \Sent
  }
  mailbox Trash {
    special_use = \Trash
  }
  prefix =
}
passdb {
  args = /etc/dovecot/dovecot-ldap.conf.ext
  driver = ldap
}
plugin {
  mail_replica = tcps::
  sieve = ~/.dovecot.sieve
  sieve_dir = ~/sieve
  sieve_extensions = +editheader +imapflags
}
postmaster_address = postmaster@
protocols = " imap sieve pop3"
replication_max_conns = 12
service aggregator {
  fifo_listener replication-notify-fifo {
    user = mail
  }
  unix_listener replication-notify {
    user = mail
  }
}
service auth {
  unix_listener /var/run/dovecot-exim-bridge {
    mode = 0660
    user = Debian-exim
  }
}
service doveadm {
  inet_listener {
    port = 
    ssl = yes
  }
}
service imap-login {
  inet_listener imap {
    port = 143
  }
  }
  inet_listener imaps {
    port = 993
    ssl = yes
  }
  process_limit = 512
  process_min_avail = 4
  service_count = 1
}
service imap {
  process_limit = 1024
}
service managesieve-login {
  inet_listener sieve {
    port = 
  }
  process_min_avail = 1
  service_count = 8
  vsz_limit = 256 M
}
service managesieve {
  process_limit = 1024
}
service replicator {
  process_min_avail = 1
  unix_listener replicator-doveadm {
    mode = 0666
  }
}
ssl = required
ssl_cert = 

Re: Deleted mail reappearing

[Sami Ketola]

> On 1. Sep 2020, at 7.28, John S  wrote:
> 
> Recently I upgraded dovecot quite a few major versions. Since then some 
> mailboxes are experiencing deleted mail reappearing. It seems to be happening 
> roughly once a day, but not for every mail client or mailbox, I have at least 
> two clients with this issue: Thunderbird and Outlook.
> 
> Storage is local under two partitions, one as alternative storage.


since you are running mdbox did you run purge before the upgrade?

Sami

Re: using %d as a variable in the ldap search base

[Luke Schierer]

On Tue, Sep 01, 2020 at 03:03:34PM +0200, Marc Roos wrote:
>  
> If you already tested %d in the ou of the auth user bind, and it is not 
> working. I guess you are just left with options like
> 
> 1. 3 different vm's
> 2. auth bind = no base, scope subtree that is parent of those 3 ou's and 
> then apply filters something like
> pass_filter = (&(objectClass=posixAccount)(uid=%n)(ou:dn:=%d))

It seems that at least some (perhaps most?) LDAP server implementations do
not allow this. Filtering on an OU is not allowed in an ldap
query by either active directory or fedora's 389-ds.  I'm not sure
about openldap.  I'm currently using fedora's 389-ds.  

> 3. investigate if nslcd(/?) has an option to use multiple queries 
> for passwd and have dovecot authenticate against the system.

This is what I'm doing as a work around while I pursue figuring out
why binding to ldap directly doesn't work.  SSSD is having no problem
with the domain substitution, but every once in a while the SSSD/PAM
integration acts up and I hate debugging PAM.  I can of course
continue this route, but I shouldn't have to.  Dovecot's documentation
says it should be able to do this.  Is there any debugging information
I can provide to help drive this towards a resolution instead of being
resigned to broken software? 

Thanks!

Luke

> 
> 
> 
> -Original Message-
> Cc: dovecot; luke-dovecot
> Subject: Re: using %d as a variable in the ldap search base
> 
> I'm trying to use auth bind to avoid having a plain text password in a 
> config file.  With %u instead of %n, the @domain part of the login ends 
> up in the uid field of the search filter.  As I said, my OUs have 
> overlapping users, so I have configured things such that the users are 
> logging in with user@domain,  and need to get the domain component into 
> the search base as you cannot filter on an Organizational Unit in an 
> ldap filter.  If I have a single search base, it will return multiple 
> users for those %ns that overlap.  If I use %u, it will return no users 
> for any account, because the uids do not have the @domain in them.  I 
> tried both ways. 
> 
> Luke
> 
> On Mon, Aug 31, 2020 at 09:45:17PM +0200, Marc Roos wrote:
> > You have two ways of authenticating against ldap. I decided to use the 
> 
> > method where a single account has access to the user credentials.
> > (Advantage of this method, you can limit ldap lookups eg do not have 
> > to do 2nd for the userdb)
> > 
> > debug_level = 1
> > uris = ldaps://ldap.local:8443
> > dn = cn=,cn=b,ou=c,dc=,dc=,dc=local
> > dnpass = 
> > base = ou=asdfadsfa,ou=,ou=ggg,dc=f,dc=,dc=local
> > scope = subtree
> > 
> > user_filter = (&(objectClass=posixAccount)(uid=%u)
> > pass_attrs =
> > uid=user,userPassword=password,host=host,homeDirectory=userdb_home,uid
> > Nu
> > mber=userdb_uid,gidNumber=userdb_gid
> > 
> > However I did not have any issues authenticating against ldap with 
> > yours also. But using the %u
> > 
> > 
> > 
> > 
> > 
> > -Original Message-
> > From: Luke Schierer [mailto:luke-dove...@schierer.org]
> > Sent: maandag 31 augustus 2020 21:32
> > To: dovecot@dovecot.org
> > Subject: using %d as a variable in the ldap search base
> > 
> > Hi,
> > 
> > I'm trying to configure dovecot to use LDAP authentication directly, 
> > and I'm having a bit of trouble.  I have a somewhat unusual setup, in 
> > that I have an LDAP directory that has 3 OUs each of which have their 
> > own set of users, some of which overlap.  As I was trying to figure 
> > things out, I was setting the search base in my 
> > /etc/dovecot/dovecot-ldap.conf.ext
> > file which is referenced by /etc/dovecot/conf.d/auth-ldap.conf.ext  
> > file to
> > 
> > base = ou=%d,dc=thecrazyguys,dc=net
> > 
> > however, the resulting searches against the directory endup just 
> > dropping the %d, resulting in
> > 
> > ou=,dc=thecrazyguys,dc=net
> > 
> > which is invalid.  On a whim, I tried a search base of
> > 
> > base = dc=%d,dc=thecrazyguys,dc=net
> > 
> > and found that it did correctly substitute in the variable, which 
> > would be correct, except that my ldap tree is set up with OUs and not 
> > an extra DC segment.
> > 
> > for whatever reason, it will do variable substitution for dc=%d, but 
> > not for ou=%d. this is certainly not documented, and seems like wrong 
> > behavior, since having an ou in a search base is valid.
> > 
> > I'm including configuration information below.  Please let me know if 
> > I've missed including information that is required.
> > 
> > Thanks!
> > Luke
> > 
> > 
> > luke@schierer@littera001:/etc/dovecot$ lsb_release -rd
> > Description: Ubuntu 18.04.5 LTS
> > Release: 18.04
> > luke@schierer@littera001:/etc/dovecot$
> > 
> > luke@schierer@littera001:/etc/dovecot$ dpkg -l | grep -i dovecot ii 
> > dovecot-core 1:2.2.33.2-1ubuntu4.6 amd64 ii dovecot-imapd
> > 1:2.2.33.2-1ubuntu4.6 amd64 ii dovecot-ldap 1:2.2.33.2-1ubuntu4.6 
> > amd64 ii dovecot-pop3d 

Re: Dovecot Proxy

[@lbutlr]

On 31 Aug 2020, at 03:33, Thoralf Rickert-Wendt  wrote:
> documentation https://wiki1.dovecot.org/HowTo/ImapProxy (which is really old 
> and should be updated)

That is documentation for Dovecott version 1 (that's the 1 in wiki1).

Other than that, I can't help you, but this documentation is aboslitley not 
relevant to version 2.x. This might help: 



(I don't know why it is Imapc Proxy, but so it goes)





-- 
"Are you pondering what I'm pondering?"
Pinky: (talking to his reflection in the mirror) Pinky, are you
pondering what I'm pondering?
Pinky's Reflection: Why, yes,
Pinky! Yes, I am! But where would you get a chicken, 20 yards of
spandex and smelling salts at this hour?

Re: Deleted mail reappearing

[@lbutlr]

On 31 Aug 2020, at 22:28, John S  wrote:
> Recently I upgraded dovecot quite a few major versions. Since then some 
> mailboxes are experiencing deleted mail reappearing. It seems to be happening 
> roughly once a day, but not for every mail client or mailbox, I have at least 
> two clients with this issue: Thunderbird and Outlook

Did you reindex everything? How many versions did you skip?

I do not think the way messages are marked has changed, but I would guess this 
is an indexing issue.



-- 
@notallmikaylas Any man who is genuinely scared that feminism means
female dominance pictures a world where men are treated like
women. Think about that.

RE: using %d as a variable in the ldap search base

[Marc Roos]

 
If you already tested %d in the ou of the auth user bind, and it is not 
working. I guess you are just left with options like

1. 3 different vm's
2. auth bind = no base, scope subtree that is parent of those 3 ou's and 
then apply filters something like
pass_filter = (&(objectClass=posixAccount)(uid=%n)(ou:dn:=%d))
3. investigate if nslcd(/?) has an option to use multiple queries 
for passwd and have dovecot authenticate against the system.



-Original Message-
Cc: dovecot; luke-dovecot
Subject: Re: using %d as a variable in the ldap search base

I'm trying to use auth bind to avoid having a plain text password in a 
config file.  With %u instead of %n, the @domain part of the login ends 
up in the uid field of the search filter.  As I said, my OUs have 
overlapping users, so I have configured things such that the users are 
logging in with user@domain,  and need to get the domain component into 
the search base as you cannot filter on an Organizational Unit in an 
ldap filter.  If I have a single search base, it will return multiple 
users for those %ns that overlap.  If I use %u, it will return no users 
for any account, because the uids do not have the @domain in them.  I 
tried both ways. 

Luke

On Mon, Aug 31, 2020 at 09:45:17PM +0200, Marc Roos wrote:
> You have two ways of authenticating against ldap. I decided to use the 

> method where a single account has access to the user credentials.
> (Advantage of this method, you can limit ldap lookups eg do not have 
> to do 2nd for the userdb)
> 
> debug_level = 1
> uris = ldaps://ldap.local:8443
> dn = cn=,cn=b,ou=c,dc=,dc=,dc=local
> dnpass = 
> base = ou=asdfadsfa,ou=,ou=ggg,dc=f,dc=,dc=local
> scope = subtree
> 
> user_filter = (&(objectClass=posixAccount)(uid=%u)
> pass_attrs =
> uid=user,userPassword=password,host=host,homeDirectory=userdb_home,uid
> Nu
> mber=userdb_uid,gidNumber=userdb_gid
> 
> However I did not have any issues authenticating against ldap with 
> yours also. But using the %u
> 
> 
> 
> 
> 
> -Original Message-
> From: Luke Schierer [mailto:luke-dove...@schierer.org]
> Sent: maandag 31 augustus 2020 21:32
> To: dovecot@dovecot.org
> Subject: using %d as a variable in the ldap search base
> 
> Hi,
> 
> I'm trying to configure dovecot to use LDAP authentication directly, 
> and I'm having a bit of trouble.  I have a somewhat unusual setup, in 
> that I have an LDAP directory that has 3 OUs each of which have their 
> own set of users, some of which overlap.  As I was trying to figure 
> things out, I was setting the search base in my 
> /etc/dovecot/dovecot-ldap.conf.ext
> file which is referenced by /etc/dovecot/conf.d/auth-ldap.conf.ext  
> file to
> 
> base = ou=%d,dc=thecrazyguys,dc=net
> 
> however, the resulting searches against the directory endup just 
> dropping the %d, resulting in
> 
> ou=,dc=thecrazyguys,dc=net
> 
> which is invalid.  On a whim, I tried a search base of
> 
> base = dc=%d,dc=thecrazyguys,dc=net
> 
> and found that it did correctly substitute in the variable, which 
> would be correct, except that my ldap tree is set up with OUs and not 
> an extra DC segment.
> 
> for whatever reason, it will do variable substitution for dc=%d, but 
> not for ou=%d. this is certainly not documented, and seems like wrong 
> behavior, since having an ou in a search base is valid.
> 
> I'm including configuration information below.  Please let me know if 
> I've missed including information that is required.
> 
> Thanks!
> Luke
> 
> 
> luke@schierer@littera001:/etc/dovecot$ lsb_release -rd
> Description: Ubuntu 18.04.5 LTS
> Release: 18.04
> luke@schierer@littera001:/etc/dovecot$
> 
> luke@schierer@littera001:/etc/dovecot$ dpkg -l | grep -i dovecot ii 
> dovecot-core 1:2.2.33.2-1ubuntu4.6 amd64 ii dovecot-imapd
> 1:2.2.33.2-1ubuntu4.6 amd64 ii dovecot-ldap 1:2.2.33.2-1ubuntu4.6 
> amd64 ii dovecot-pop3d 1:2.2.33.2-1ubuntu4.6 amd64 
> luke@schierer@littera001:/etc/dovecot$
> 
> root@littera001:/etc/dovecot# dovecot -n # 2.2.33.2 (d6601f4ec): 
> /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.21 (92477967) # OS: 

> Linux 4.15.0-112-generic x86_64 Ubuntu 18.04.5 LTS auth_verbose = yes 
> first_valid_uid = 1001 imap_hibernate_timeout = 100 secs lock_method = 

> dotlock mail_cache_min_mail_count = 15 mail_fsync = always 
> mail_location = maildir:~/Maildir:LAYOUT=fs:INDEX=MEMORY
> mail_nfs_index = yes
> mail_nfs_storage = yes
> mail_privileged_group = mail
> mailbox_list_index = yes
> maildir_broken_filename_sizes = yes
> maildir_very_dirty_syncs = yes
> mbox_min_index_size = 10 B
> mmap_disable = yes
> namespace inbox {
>   inbox = yes
>   location =
>   mailbox Drafts {
> special_use = \Drafts
>   }
>   mailbox Junk {
> special_use = \Junk
>   }
>   mailbox Sent {
> special_use = \Sent
>   }
>   mailbox "Sent Messages" {
> special_use = \Sent
>   }
>   mailbox Trash {
> special_use = \Trash
>   }
>   prefix =
>   separator = /
> }
> passdb {

Re: Bug: Dovecot appending "MISSING_DOMAIN" to fetch envelope responses

[@lbutlr]

On 31 Aug 2020, at 06:23, Michael Gratton  wrote:
> Any word about this issue? Should I file a bug in an actual bug tracker or 
> something?

I don't think it's a bug?



-- 
Eliot: We'll figure it out. We always do.
Margo: When it's a test to cheat on. Not when we're stuck in some epic fantasy
   that likes to behead heroes halfway through season one. If we even are
   heroes. We might be comic relief.

Re: Dovecot Proxy

[Thoralf Rickert-Wendt]

Hi Philon,

now, it's time for "Mahlzeit" ;-)

Sorry, that I read the wiki1 instead of wiki2. I thought the 1 means 
that it is server one of ... my fault. Also not reading the first line 
above the menu. My focus was really on the content. ;-)


Also my problem with the doc of Dovecot2 proxy is, that the document 
https://doc.dovecot.org/configuration_manual/authentication/proxies/ has 
less details for a domain only example. That works as in the Dovecot1 
doc, but it isn't documented anymore. Also the location under 
"authentication" chapter in the Wiki didn't tell me, that this is the 
"new Dovecot proxy documentation". I thought, this was only related to 
authentication issues. I would recommend to either restructure the 
wiki2, that it makes it more clear to the user or make some notes on 
https://doc.dovecot.org/admin_manual/dovecot_proxy/ and link to the 
passdb setting on 
https://doc.dovecot.org/configuration_manual/forwarding_parameters/ and 
https://doc.dovecot.org/configuration_manual/authentication/proxies/. 
Maybe there are other documents related to Proxy too, like the SNI 
settings etc. But maybe I'm the only one on the planet, that tries to 
use that. It feels a little bit like that.


The Director would be interesting, if all the mailservers in the backend 
would know each other. But thats not the case. Mailserver A and 
Mailserver B are hosting complete different domains with a complete 
different user list and complete different user admins, etc. Also 
mailcow doesnt enabled the director. So it will not help much. But it 
could be interesting, if I have multiple proxies.


Yes, the submission service inside Dovecot is there. And I tried to 
avoid to install multiple "programs" and if there is one "program" that 
handles it all, why don't use it. And I'd like to quote the first line 
of the Dovecot proxy doc: "Dovecot supports proxying IMAP, POP3, 
Submission Server 
, 
LMTP Server 
, 
and Pigeonhole ManageSieve Server 
 
connections to other hosts.".


Also I tried to open the Dovecot authentication mechanism for postfix 
(for submission) with



service auth {
  user = root
  unix_listener /var/spool/postfix/private/auth {
    group = postfix
    mode = 0660
    user = postfix
  }
}

And on Postfix part with

smtpd_sasl_auth_enabled = yes
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth

But the postfix login is always accepted (even with wrong passwords) and 
after I start to write a Mail the connection get lost after RCPT 
command. There is another problem. Before I infestigate it, I would try 
my luck with Dovecot. It is already asking the correct backend 
submission server but with SSL on a non-SSL port.


So - someone included the Submission protocol in Dovecot and someone 
wrote, that the submission could be proxied - but - its not completly 
documented or "it doesn't work" within a SSL environment. I searched for 
a simple example, where IMAP and POP3 are proxied via SSL and Submission 
too (which would mean, that Dovecot submission listens on 465) or via 
STARTTLS on 587 and redirecting it also to STARTTLS/587. But I didn't 
find anything. Also the submission documentation doesnt help, because I 
cant see any line of configuration file in it.


Ok, but first - lunchtime.

bye
Thoralf

Am 01.09.20 um 09:43 schrieb Philon:

Hi Thoralf,

I’d say first of all you should read the current docs for 2.x not the archived 
stuff. —> https://wiki2.dovecot.org/ - (It’s even mentioned in bold in the 
header)

Then to front multiple backends perhaps you want to take a look at Dovecot 
Director. —> https://wiki2.dovecot.org/Director

About SMTP I’m not sure why you would want to rely on Dovecot for that. I only 
do Postfix with Dovecot as auth backend so they can share passdb access. When 
you have 465 set up it is no big deal to also enable 587 in Postfixs master.cf.

If you want to keep Dovecot for Submission you can check the latest docs for 
Dovecot submission service: 
https://doc.dovecot.org/admin_manual/submission_server/. It has a relay server 
option with port. Also settings for STARTTLS etcpp can be found there.


Mahlzeit!

Philon


On 31 Aug 2020, at 11:33, Thoralf Rickert-Wendt  wrote:

Hello everyone,

it's my first post here on this mailing list and I hope, I make it right.

I posted a question on 
https://serverfault.com/questions/1031441/dovecot-as-proxy-with-submission and 
nobody was able to answer it. So I decided to push that question here (I'm 
talking about any new dovecot version and I've tested it with 2.3.4.1 
(f79e8e7e4)).

I try to run a dovecot proxy in front of a big number of mail servers (serving SMTP-in, 
submission, IMAP, POP3, Sieve). I need that proxy, because I run out of IPv4 addresses. 
Of 

Re: using %d as a variable in the ldap search base

[Luke Schierer]

I'm trying to use auth bind to avoid having a plain text password in a
config file.  With %u instead of %n, the @domain part of the login
ends up in the uid field of the search filter.  As I said, my OUs have
overlapping users, so I have configured things such that the users are
logging in with user@domain,  and need to get the domain component
into the search base as you cannot filter on an Organizational Unit
in an ldap filter.  If I have a single search base, it will return
multiple users for those %ns that overlap.  If I use %u, it will
return no users for any account, because the uids do not have the
@domain in them.  I tried both ways. 

Luke

On Mon, Aug 31, 2020 at 09:45:17PM +0200, Marc Roos wrote:
> You have two ways of authenticating against ldap. I decided to use the 
> method where a single account has access to the user credentials. 
> (Advantage of this method, you can limit ldap lookups eg do not have to 
> do 2nd for the userdb)
> 
> debug_level = 1
> uris = ldaps://ldap.local:8443
> dn = cn=,cn=b,ou=c,dc=,dc=,dc=local
> dnpass = 
> base = ou=asdfadsfa,ou=,ou=ggg,dc=f,dc=,dc=local
> scope = subtree
> 
> user_filter = (&(objectClass=posixAccount)(uid=%u)
> pass_attrs = 
> uid=user,userPassword=password,host=host,homeDirectory=userdb_home,uidNu
> mber=userdb_uid,gidNumber=userdb_gid
> 
> However I did not have any issues authenticating against ldap with yours 
> also. But using the %u
> 
> 
> 
> 
> 
> -Original Message-
> From: Luke Schierer [mailto:luke-dove...@schierer.org] 
> Sent: maandag 31 augustus 2020 21:32
> To: dovecot@dovecot.org
> Subject: using %d as a variable in the ldap search base
> 
> Hi,
> 
> I'm trying to configure dovecot to use LDAP authentication directly, and 
> I'm having a bit of trouble.  I have a somewhat unusual setup, in that I 
> have an LDAP directory that has 3 OUs each of which have their own set 
> of users, some of which overlap.  As I was trying to figure things out, 
> I was setting the search base in my /etc/dovecot/dovecot-ldap.conf.ext 
> file which is referenced by /etc/dovecot/conf.d/auth-ldap.conf.ext  file 
> to
> 
> base = ou=%d,dc=thecrazyguys,dc=net
> 
> however, the resulting searches against the directory endup just 
> dropping the %d, resulting in
> 
> ou=,dc=thecrazyguys,dc=net 
> 
> which is invalid.  On a whim, I tried a search base of
> 
> base = dc=%d,dc=thecrazyguys,dc=net
> 
> and found that it did correctly substitute in the variable, which would 
> be correct, except that my ldap tree is set up with OUs and not an extra 
> DC segment.
> 
> for whatever reason, it will do variable substitution for dc=%d, but not 
> for ou=%d. this is certainly not documented, and seems like wrong 
> behavior, since having an ou in a search base is valid.
> 
> I'm including configuration information below.  Please let me know if 
> I've missed including information that is required. 
> 
> Thanks!
> Luke
> 
> 
> luke@schierer@littera001:/etc/dovecot$ lsb_release -rd
> Description: Ubuntu 18.04.5 LTS
> Release: 18.04
> luke@schierer@littera001:/etc/dovecot$
> 
> luke@schierer@littera001:/etc/dovecot$ dpkg -l | grep -i dovecot ii 
> dovecot-core 1:2.2.33.2-1ubuntu4.6 amd64 ii dovecot-imapd 
> 1:2.2.33.2-1ubuntu4.6 amd64 ii dovecot-ldap 1:2.2.33.2-1ubuntu4.6 amd64 
> ii dovecot-pop3d 1:2.2.33.2-1ubuntu4.6 amd64 
> luke@schierer@littera001:/etc/dovecot$
> 
> root@littera001:/etc/dovecot# dovecot -n # 2.2.33.2 (d6601f4ec): 
> /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.21 (92477967) # OS: 
> Linux 4.15.0-112-generic x86_64 Ubuntu 18.04.5 LTS auth_verbose = yes 
> first_valid_uid = 1001 imap_hibernate_timeout = 100 secs lock_method = 
> dotlock mail_cache_min_mail_count = 15 mail_fsync = always mail_location 
> = maildir:~/Maildir:LAYOUT=fs:INDEX=MEMORY
> mail_nfs_index = yes
> mail_nfs_storage = yes
> mail_privileged_group = mail
> mailbox_list_index = yes
> maildir_broken_filename_sizes = yes
> maildir_very_dirty_syncs = yes
> mbox_min_index_size = 10 B
> mmap_disable = yes
> namespace inbox {
>   inbox = yes
>   location =
>   mailbox Drafts {
> special_use = \Drafts
>   }
>   mailbox Junk {
> special_use = \Junk
>   }
>   mailbox Sent {
> special_use = \Sent
>   }
>   mailbox "Sent Messages" {
> special_use = \Sent
>   }
>   mailbox Trash {
> special_use = \Trash
>   }
>   prefix =
>   separator = /
> }
> passdb {
>   args = /etc/dovecot/dovecot-ldap.conf.ext
>   driver = ldap
> }
> protocols = " imap pop3"
> ssl_cert =  ssl_key =  # hidden, use -P to show it
> userdb {
>   args = /etc/dovecot/dovecot-ldap.conf.ext
>   driver = ldap
> }
> protocol imap {
>   mail_max_userip_connections = 20
> }
> root@littera001:/etc/dovecot# 
> 
> root@littera001:/etc/dovecot# cat /etc/dovecot/dovecot-ldap.conf.ext | 
> grep -v ^# | uniq | more
> 
> hosts = censor001.plerumque.thecrazyguys.net
> 
> tls = yes
> tls_require_cert = allow
> 
> debug_level = 4
> 
> auth_bind = yes
> 
> base = 

AW: replicator: Panic: data stack: Out of memory

[Thomas Tsiakalakis]

I just found this post from 2017: 
http://dovecot.2317879.n4.nabble.com/replicator-crashing-oom-td59402.html
I removed the replication_sync_timeout setting and now it's working fine.


Thomas Tsiakalakis

Team Applikationsbetrieb
GDV Dienstleistungs-GmbH
Tel: +49(40)33449-4318
E-Mail: mailto:thomas.tsiakala...@gdv-dl.de

GDV Dienstleistungs-GmbH
Glockengießerwall 1
D-20095 Hamburg
www.gdv-dl.de

Niederlassungen:

Wilhelmstraße 43 / 43 G
10117 Berlin

Frankenstraße 18a
20097 Hamburg

Sitz und Registergericht: Hamburg
HRB 145291
USt.-IdNr : DE 205183123

Geschäftsführer:
Dr. Jens Bartenwerfer
Fred di Giuseppe Chiachiarella

Aufsichtsratsvorsitzender: Werner Schmidt

--
Diese E-Mail und alle Anhänge enthalten vertrauliche und/oder rechtlich 
geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese 
E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und 
vernichten Sie diese E-Mail. Das unerlaubte Kopieren sowie die unbefugte 
Weitergabe der E-Mail ist nicht gestattet.

This e-mail and any attached files may contain confidential and/or privileged 
information. If you are not the intended recipient (or have received this 
e-mail in error) please notify the sender immediately and destroy this e-mail. 
Any unauthorised copying, disclosure or distribution of the material in this 
e-mail is strictly forbidden.

-Ursprüngliche Nachricht-
Von: Aki Tuomi 
Gesendet: Freitag, 14. August 2020 12:21
An: Thomas Tsiakalakis ; dovecot@dovecot.org
Betreff: Re: replicator: Panic: data stack: Out of memory

Try setting

service replicator {
  vsz_limit = 0
}

Aki

> On 14/08/2020 13:19 Thomas Tsiakalakis  wrote:
>
>
>
> So nobody has any idea why this could happen?
> Let me know if I should provide more Information Thanks
>
>  ThomasTsiakalakis
>
> Team Applikationsbetrieb
>  GDV Dienstleistungs-GmbH
>  Tel: +49(40)33449-4318
>  Fax:
>  E-Mail:thomas.tsiakala...@gdv-dl.de
>
>
>
>
>  GDV Dienstleistungs-GmbH
>  Glockengießerwall 1
>  D-20095 Hamburg
>  www.gdv-dl.de (http://www.gdv-dl.de)
>
>  Niederlassungen:
>
>  Wilhelmstraße 43 / 43 G
>  10117 Berlin
>
>  Frankenstraße 18
>  20097 Hamburg
>
>  Sitz und Registergericht: Hamburg
>  HRB 145291
>  USt.-IdNr : DE 205183123
>
>  Geschäftsführer:
>  Dr. Jens Bartenwerfer
>  Fred di Giuseppe Chiachiarella
>
>  Aufsichtsratsvorsitzender: Werner Schmidt
>
>  --
>  Diese E-Mail und alle Anhänge enthalten vertrauliche und/oder rechtlich 
> geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder 
> diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den 
> Absender und vernichten Sie diese E-Mail. Das unerlaubte Kopieren sowie die 
> unbefugte Weitergabe der E-Mail ist nicht gestattet.
>
>  This e-mail and any attached files may contain confidential and/or 
> privileged information. If you are not the intended recipient (or have 
> received this e-mail in error) please notify the sender immediately and 
> destroy this e-mail. Any unauthorised copying, disclosure or distribution of 
> the material in this e-mail is strictly forbidden.
>
>
> Von:Thomas Tsiakalakis
>  Gesendet: Dienstag, 23. Juni 2020 11:25
>  An: dovecot@dovecot.org
>  Betreff: AW: replicator: Panic: data stack: Out of memory I managed
> to convince the System that I really want a core dump Von:Thomas
> Tsiakalakis
>  Gesendet: Donnerstag, 18. Juni 2020 16:26
>  An: 'dovecot@dovecot.org' 
>  Betreff: replicator: Panic: data stack: Out of memory Hi, I have set
> up 2 SLES 15 Hosts with dovecot and replication. Everything seems to work 
> fine but every time a message is delivered, I get an out of memory error in 
> the logs. The Replication itself seems to work fine though.
> I increased default_vsz_limit to 512M but the only thing that changed
> was that dovecot was trying to allocate 1073741864 bytes instead of
> 268435496 As I said, I’m running SLES 15 SP1 and Dovecot 2.3.10 (0da0eff44) 
> (both Hosts have the same version) Each Host currently has 8GB of Memory.
> # free -h
> total used free shared buff/cache available
> Mem: 7.8Gi 278Mi 7.3Gi 47Mi 183Mi 7.2Gi
> Swap: 4.0Gi 0B 4.0Gi
> # journalctl -f
> Jun 18 15:55:48 mail1 postfix/pickup[3457]: 18533C009C8: uid=0
> from= Jun 18 15:55:48 mail1 postfix/cleanup[3669]: 18533C009C8:
> message-id=<20200618135548.18533C009C8@mail1>
> Jun 18 15:55:48 mail1 postfix/qmgr[3458]: 18533C009C8:
> from=, size=431, nrcpt=1 (queue active) Jun 18 15:55:48
> mail1 dovecot[1833]: lmtp(3673): Connect from local Jun 18 15:55:48
> mail1 dovecot[1833]: replicator: Panic: data stack: Out of memory when
> allocating 268435496 bytes Jun 18 15:55:48 mail1 dovecot[1833]: replicator: 
> Error: Raw backtrace: 
> /usr/lib64/dovecot/libdovecot.so.0(backtrace_append+0x42) [0x7f8346d6d262] -> 
> /usr/lib64/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7f8346d6d37e] -> 
> 

AW: replicator: Panic: data stack: Out of memory

[Thomas Tsiakalakis]

Still the same error just with more bytes:

replicator: Panic: data stack: Out of memory when allocating 17179869224 bytes


Thomas Tsiakalakis

Team Applikationsbetrieb
GDV Dienstleistungs-GmbH
Tel: +49(40)33449-4318
E-Mail: mailto:thomas.tsiakala...@gdv-dl.de

GDV Dienstleistungs-GmbH
Glockengießerwall 1
D-20095 Hamburg
www.gdv-dl.de

Niederlassungen:

Wilhelmstraße 43 / 43 G
10117 Berlin

Frankenstraße 18a
20097 Hamburg

Sitz und Registergericht: Hamburg
HRB 145291
USt.-IdNr : DE 205183123

Geschäftsführer:
Dr. Jens Bartenwerfer
Fred di Giuseppe Chiachiarella

Aufsichtsratsvorsitzender: Werner Schmidt

--
Diese E-Mail und alle Anhänge enthalten vertrauliche und/oder rechtlich 
geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese 
E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und 
vernichten Sie diese E-Mail. Das unerlaubte Kopieren sowie die unbefugte 
Weitergabe der E-Mail ist nicht gestattet.

This e-mail and any attached files may contain confidential and/or privileged 
information. If you are not the intended recipient (or have received this 
e-mail in error) please notify the sender immediately and destroy this e-mail. 
Any unauthorised copying, disclosure or distribution of the material in this 
e-mail is strictly forbidden.

-Ursprüngliche Nachricht-
Von: Aki Tuomi 
Gesendet: Freitag, 14. August 2020 12:21
An: Thomas Tsiakalakis ; dovecot@dovecot.org
Betreff: Re: replicator: Panic: data stack: Out of memory

Try setting

service replicator {
  vsz_limit = 0
}

Aki

> On 14/08/2020 13:19 Thomas Tsiakalakis  wrote:
>
>
>
> So nobody has any idea why this could happen?
> Let me know if I should provide more Information Thanks
>
>  ThomasTsiakalakis
>
> Team Applikationsbetrieb
>  GDV Dienstleistungs-GmbH
>  Tel: +49(40)33449-4318
>  Fax:
>  E-Mail:thomas.tsiakala...@gdv-dl.de
>
>
>
>
>  GDV Dienstleistungs-GmbH
>  Glockengießerwall 1
>  D-20095 Hamburg
>  www.gdv-dl.de (http://www.gdv-dl.de)
>
>  Niederlassungen:
>
>  Wilhelmstraße 43 / 43 G
>  10117 Berlin
>
>  Frankenstraße 18
>  20097 Hamburg
>
>  Sitz und Registergericht: Hamburg
>  HRB 145291
>  USt.-IdNr : DE 205183123
>
>  Geschäftsführer:
>  Dr. Jens Bartenwerfer
>  Fred di Giuseppe Chiachiarella
>
>  Aufsichtsratsvorsitzender: Werner Schmidt
>
>  --
>  Diese E-Mail und alle Anhänge enthalten vertrauliche und/oder rechtlich 
> geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder 
> diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den 
> Absender und vernichten Sie diese E-Mail. Das unerlaubte Kopieren sowie die 
> unbefugte Weitergabe der E-Mail ist nicht gestattet.
>
>  This e-mail and any attached files may contain confidential and/or 
> privileged information. If you are not the intended recipient (or have 
> received this e-mail in error) please notify the sender immediately and 
> destroy this e-mail. Any unauthorised copying, disclosure or distribution of 
> the material in this e-mail is strictly forbidden.
>
>
> Von:Thomas Tsiakalakis
>  Gesendet: Dienstag, 23. Juni 2020 11:25
>  An: dovecot@dovecot.org
>  Betreff: AW: replicator: Panic: data stack: Out of memory I managed
> to convince the System that I really want a core dump Von:Thomas
> Tsiakalakis
>  Gesendet: Donnerstag, 18. Juni 2020 16:26
>  An: 'dovecot@dovecot.org' 
>  Betreff: replicator: Panic: data stack: Out of memory Hi, I have set
> up 2 SLES 15 Hosts with dovecot and replication. Everything seems to work 
> fine but every time a message is delivered, I get an out of memory error in 
> the logs. The Replication itself seems to work fine though.
> I increased default_vsz_limit to 512M but the only thing that changed
> was that dovecot was trying to allocate 1073741864 bytes instead of
> 268435496 As I said, I’m running SLES 15 SP1 and Dovecot 2.3.10 (0da0eff44) 
> (both Hosts have the same version) Each Host currently has 8GB of Memory.
> # free -h
> total used free shared buff/cache available
> Mem: 7.8Gi 278Mi 7.3Gi 47Mi 183Mi 7.2Gi
> Swap: 4.0Gi 0B 4.0Gi
> # journalctl -f
> Jun 18 15:55:48 mail1 postfix/pickup[3457]: 18533C009C8: uid=0
> from= Jun 18 15:55:48 mail1 postfix/cleanup[3669]: 18533C009C8:
> message-id=<20200618135548.18533C009C8@mail1>
> Jun 18 15:55:48 mail1 postfix/qmgr[3458]: 18533C009C8:
> from=, size=431, nrcpt=1 (queue active) Jun 18 15:55:48
> mail1 dovecot[1833]: lmtp(3673): Connect from local Jun 18 15:55:48
> mail1 dovecot[1833]: replicator: Panic: data stack: Out of memory when
> allocating 268435496 bytes Jun 18 15:55:48 mail1 dovecot[1833]: replicator: 
> Error: Raw backtrace: 
> /usr/lib64/dovecot/libdovecot.so.0(backtrace_append+0x42) [0x7f8346d6d262] -> 
> /usr/lib64/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7f8346d6d37e] -> 
> /usr/lib64/dovecot/libdovecot.so.0(+0xebcee) [0x7f8346d77cee] -> 
> 

shutdown_clients ignored on Director upgrade/restart

[Tom Sommer]

When yum updates a Dovecot Director, it restarts all processes - however 
a few imap-login and pop3-login processes hang because they are still 
proxying connections.


It's like the now orphaned processes are stalled, and needs a kill -9 to 
die.


This is a problem because those stalled connections are still connected 
to backends, so when new connections are made from the same account, 
they do not end up on the same backend and thus index corruption happens 
(same account, two different backends).


shutdown_clients is enabled but it appears Dovecot restarts regardless 
of verifying all processes/clients are shut down.


Just had this happen upon yum update from dovecot-2.3.11.3-3 to 
dovecot-2.3.11.3-4


--
Tom

Re: about header address parsing

[Timo Sirainen]

On 1. Sep 2020, at 6.24, TACHIBANA Masashi  wrote:
> 
> Hi,
> 
> Is this expected or not?
> 
> From: us...@fuga.example.com 
> To: us...@hoge.example.com 
> ↓
> a uid fetch 43055 (envelope)
> * 1860 FETCH (UID 43055 ENVELOPE ("Thu, 30 Jul 2020 13:52:59 +0900" "test1" 
> ((NIL NIL "user1" "fuga.example.com")) ((NIL NIL "user1" "fuga.example.com")) 
> ((NIL NIL "user1" "fuga.example.com")) ((NIL NIL "user2" "hoge.example.com")) 
> NIL NIL NIL ""))

This is an invalid email address, so it's neither correct nor incorrect to have 
this output. But this reminded me that I was going to discuss about this with 
other IETF people. Lets see what others think: 
https://mailarchive.ietf.org/arch/msg/extra/sqRTdsV_DGBhHu2ghdCDFo_pM8Q/ 

> From: "us...@fuga.example.com" 
> To: "us...@hoge.example.com" 
> ↓
> a uid fetch 43056 (envelope)
> * 1861 FETCH (UID 43056 ENVELOPE ("Thu, 30 Jul 2020 13:53:59 +0900" "test1" 
> (("us...@fuga.example.com" NIL "user1" "example.com")) 
> (("us...@fuga.example.com" NIL "user1" "example.com")) 
> (("us...@fuga.example.com" NIL "user1" "example.com")) 
> (("us...@hoge.example.com" NIL "user2" "example.com")) NIL NIL NIL 
> ""))

This is a valid email address, and ENVELOPE reply is correct.

Re: Dovecot Proxy

[Philon]

Hi Thoralf,

I’d say first of all you should read the current docs for 2.x not the archived 
stuff. —> https://wiki2.dovecot.org/ - (It’s even mentioned in bold in the 
header)

Then to front multiple backends perhaps you want to take a look at Dovecot 
Director. —> https://wiki2.dovecot.org/Director

About SMTP I’m not sure why you would want to rely on Dovecot for that. I only 
do Postfix with Dovecot as auth backend so they can share passdb access. When 
you have 465 set up it is no big deal to also enable 587 in Postfixs master.cf.

If you want to keep Dovecot for Submission you can check the latest docs for 
Dovecot submission service: 
https://doc.dovecot.org/admin_manual/submission_server/. It has a relay server 
option with port. Also settings for STARTTLS etcpp can be found there.


Mahlzeit!

Philon

> On 31 Aug 2020, at 11:33, Thoralf Rickert-Wendt  wrote:
> 
> Hello everyone,
> 
> it's my first post here on this mailing list and I hope, I make it right.
> 
> I posted a question on 
> https://serverfault.com/questions/1031441/dovecot-as-proxy-with-submission 
> and nobody was able to answer it. So I decided to push that question here 
> (I'm talking about any new dovecot version and I've tested it with 2.3.4.1 
> (f79e8e7e4)).
> 
> I try to run a dovecot proxy in front of a big number of mail servers 
> (serving SMTP-in, submission, IMAP, POP3, Sieve). I need that proxy, because 
> I run out of IPv4 addresses. Of course I use IPv6 too, but many customers 
> still have problems with there providers and they really don't want to share 
> their mails on a "shared-mailserver". I planed to use Dovecot for IMAPS, 
> POP3S, SMTP-submission(465) and postfix for the rest. If I find a solution 
> for sieve, I would try that too, but that is very optional.
> 
> With the documentation https://wiki1.dovecot.org/HowTo/ImapProxy (which is 
> really old and should be updated) and some other ascii docs (from an Apple 
> mirror somewhere deep in the web) I was able to build a IMAP/POP3 proxy that 
> forwards requests from outside to a specific backend using SSL (993,995). 
> That works - I think.You can find the config on the serverfault page.
> 
> In general - all known domains in backend are using SSL and the passdb 
> forwards all requests to the backend via SSL. So - I understand:||
> 
> |password_query =
>   SELECT
> NULL AS password,
> NULL AS destuser,
> host,
> 'Y' AS nologin,
> 'Y' AS nodelay,
> 'Y' AS nopassword,
> 'Y' AS proxy,
> 'any-cert' AS `ssl`
>   FROM
> proxy_domain
>   WHERE
> domain = '%d' |
> 
> But that is only 50% of the show. The rest ist submission (and maybe sieve). 
> Practically the submission implementation in dovecot works too. But because 
> dovecot by default only opens port 587 (starttls), my passdb setting has a 
> problem.
> 
> When I try to use that port Dovecot tries to use SSL on the backend/587 too - 
> but that is wrong (it should either use 465 or should try to use starttls).
> 
> So, I have the following options.
> 
> - find a way to configure dovecot-proxy to listen on 465 with SSL for 
> submission service and hope that it uses the same port
>   - but I didn't find any documentation for that and need help
> 
> - find a way to configure dovecot-proxy/passdb to return starttls=y when 
> dovecot-submission is used (use a different passdb)
>   - but I didn't find any documentation for that and I'm not sure, if this 
> worls on service/protocol level
> 
> - find a way to configure the passdb answer based on the used port/protocol. 
> But I only know the parameter %u, %d and %p.
>   - so it would be nice to find a way to also select the protocol (if already 
> developed)
> 
> - find a way to make a patch in dovecot (which isn't easy for me, because I 
> don't really know the code)
> 
> Has somebody an idea, how I can configure the dovecot-proxy in that way.
> 
> bye
> Thoralf
> 
>