Re: maildirfolder file created in maildir root during auto-creation with 2.3.4.1 but not 2.2.27
Hello,
On Wed, 5 Feb 2020 08:58:29 +0200 Aki Tuomi wrote:
> Can you provide full doveconf -n output? Also how are you delivering mail?
>
As pretty much implied, Exim is delivering mails, w/o problems.
And if it gets to create the home directory, everything is fine
and maildirsize gets put there.
But if the first access is via the newer dovecot the bogus maildirfolder
file gets created in the home directory and prevents Exim (and itself?)
from putting a maildirsize there.
My bet is that that something in the auto-create logic changed or the
"mail_home" needing to be set explicitly instead of defaulting to
mail_location if unset, etc.
Redacted and relevant parts only:
---
# 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.4 ()
# OS: Linux 4.19.0-6-amd64 x86_64 Debian 10.2
# Hostname: testbox.gol.com
auth_default_realm = gol.com
default_client_limit = 16384
default_process_limit = 1024
first_valid_uid = 8
imap_hibernate_timeout = 30 secs
imap_idle_notify_interval = 8 mins
imap_logout_format = in=%i out=%o head=<%{fetch_hdr_count}> del=<%{deleted}>
exp=<%{expunged}> trash=<%{trashed}> session=<%{session}>
login_trusted_networks = some.net.work
mail_gid = 8
mail_location = maildir:%h
mail_privileged_group = mail
mail_uid = 8
mailbox_idle_check_interval = 1 mins
maildir_very_dirty_syncs = yes
passdb {
args = /etc/dovecot/dovecot-ldap.conf.ext
driver = ldap
}
plugin {
quota = maildir:User
quota_rule = ?:storage=200M
quota_rule2 = Trash:storage=+50M
sieve = file:~/sieve;active=~/.dovecot.sieve
}
userdb {
args = /etc/dovecot/dovecot-ldap.conf.ext
driver = ldap
}
verbose_proctitle = yes
protocol imap {
mail_max_userip_connections = 40
mail_plugins = quota imap_quota
}
protocol pop3 {
mail_plugins = quota
}
---
Regards,
Christian
> Aki
>
> On 5.2.2020 4.24, Christian Balzer wrote:
> >
> > Hello,
> >
> > as the tin says.
> > I have several servers running 2.2.27 (Debian stretch) and am adding new
> > ones with 2.3.4.1 (Debian buster).
> > The configs were upgraded where needed but neither 10-mail.conf nor
> > 15-mailboxes.conf were changed.
> > 15-mailboxes is all commented out (I guess the default is auto-create,
> > which isn't documented anywhere I could find) and the only non-comments in
> > 10-mail.conf are
> > ---
> > mail_location = maildir:%h
> > mail_privileged_group = mail
> > ---
> >
> > So yes, no namespaces are explicitly defined/declared.
> >
> >
> > The 2.3.4.1 version wrongly creates a maildirfolder file in the home
> > directory (maildir root), preventing exim from correctly creating/using
> > maildirsize.
> >
> > a) Is this expected behavior and can it be changed?
> > b) How can I disable inbox auto-creation if a) doesn't pan out?
> >
> > Thanks,
> >
> > Christian
>
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Rakuten Mobile Inc.
Re: auth-client via SSL?
On 4.2.2020 13.46, Heiko Schlittermann wrote:
> Hi, I'm resending this message, still hoping for an answer.
>
> Hello,
>
> does dovecot support tls-on-connect for AF INET based auth-client
> sockets?
>
> Rationale behind my question:
>
> Exim can use the Dovecot auth-client socket to delegate the
> SMTP-AUTH authentication to Dovecot.
>
> Currently Exim supports the AF UNIX only for this socket. Jeremy makes
> progress in extending this to use AF INET sockets too.
>
> While it works with clear text communication already, during testing I
> was to setup the auch-client socket as an TLS server (tls-on-connect).
> It doesn't seem to work as I'd expect. The socket still offers
> clear-text only.
>
> Here my configuration snippets regarding this socket
>
>ssl = yes
>ssl_cert = ssl_key =
> service auth {
> …
> unix_listener auth-client {
> group = _exim
> mode = 0660
> }
> inet_listener auth-client {
> name = exim
> port = 4711
> ssl = yes
> }
> }
>
> SSL connections to :993 work as expected.
>
> Best regards from Dresden/Germany
> Viele Grüße aus Dresden
> Heiko Schlittermann
Hi!
This is not (yet) implemented. You can probably workaround with haproxy
/ stunnel for now.
Aki
Re: maildirfolder file created in maildir root during auto-creation with 2.3.4.1 but not 2.2.27
Can you provide full doveconf -n output? Also how are you delivering mail? Aki On 5.2.2020 4.24, Christian Balzer wrote: > > Hello, > > as the tin says. > I have several servers running 2.2.27 (Debian stretch) and am adding new > ones with 2.3.4.1 (Debian buster). > The configs were upgraded where needed but neither 10-mail.conf nor > 15-mailboxes.conf were changed. > 15-mailboxes is all commented out (I guess the default is auto-create, > which isn't documented anywhere I could find) and the only non-comments in > 10-mail.conf are > --- > mail_location = maildir:%h > mail_privileged_group = mail > --- > > So yes, no namespaces are explicitly defined/declared. > > > The 2.3.4.1 version wrongly creates a maildirfolder file in the home > directory (maildir root), preventing exim from correctly creating/using > maildirsize. > > a) Is this expected behavior and can it be changed? > b) How can I disable inbox auto-creation if a) doesn't pan out? > > Thanks, > > Christian
maildirfolder file created in maildir root during auto-creation with 2.3.4.1 but not 2.2.27
Hello, as the tin says. I have several servers running 2.2.27 (Debian stretch) and am adding new ones with 2.3.4.1 (Debian buster). The configs were upgraded where needed but neither 10-mail.conf nor 15-mailboxes.conf were changed. 15-mailboxes is all commented out (I guess the default is auto-create, which isn't documented anywhere I could find) and the only non-comments in 10-mail.conf are --- mail_location = maildir:%h mail_privileged_group = mail --- So yes, no namespaces are explicitly defined/declared. The 2.3.4.1 version wrongly creates a maildirfolder file in the home directory (maildir root), preventing exim from correctly creating/using maildirsize. a) Is this expected behavior and can it be changed? b) How can I disable inbox auto-creation if a) doesn't pan out? Thanks, Christian -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Rakuten Mobile Inc.
Re: Strategy for fts
Am 04.02.20 um 12:37 schrieb Peter Chiochetti: Am 04.02.20 um 11:46 schrieb Francis Augusto Medeiros-Logeay: Hi Philon, Thanks a lot for your thoughts! Can I ask you if using Solr improved things for you? I have a mailbox with 15 years of e-mail and searching things take a long time. Here, SOLR itself searches a quarter million mails in split seconds and returns very good results. That is on a low memory average machine. Looking at the facts, it is closer to half a million mails in a 160GB Maildir, lots of trash too, but no one to sort it out. SOLR index is 1.2 GB in size on disk. A tremendous ratio IMO. In dovecot terms this is likely considered a small installation. We are a small team too :) and quite happy with the generous gift of dovecot, and Thunderbird BTW. Only problem is search through e.g. nested folders from IMAP: something like ESEARCH would be nice - https://tools.ietf.org/html/rfc6237 PS: There is powerful client side search in some MUAs, yet sometimes serverside comes handy. -- peter
Submission service, XCLIENT and HELO
Hello, I'm using version 2.3.4.1 and I have a fairly simple setup based on a submission server where I run dovecot and a relay server with postfix. The relevant part of dovecot's config is as follows (sanitized): hostname = submission.domain.local submission_client_workarounds = whitespace-before-path submission_relay_host = 192.168.1.1 <- postfix submission_relay_port = 25 submission_relay_trusted = yes Postfix is configured as follows: smtpd_authorized_xclient_hosts = 192.168.1.2 <- submission server Everything apparently works as expected, meaning that I can correctly authenticate on the submission server and the email is relayed to postfix, where I can see the original MUA's IP correctly logged. There is, however, a little problem with the client HELO string that is not being forwarded to postfix. Instead of having the MUA HELO, I see the submission server HELO, as shown in the following tcpdump taken on the postfix server: 220 postfix.domain.local ESMTP Postfix (Ubuntu) EHLO submission.domain.local 250-postfix.domain.local 250-PIPELINING 250-SIZE 1024 250-VRFY 250-ETRN 250-STARTTLS 250-AUTH DIGEST-MD5 CRAM-MD5 NTLM PLAIN LOGIN 250-AUTH=DIGEST-MD5 CRAM-MD5 NTLM PLAIN LOGIN 250-XCLIENT NAME ADDR PROTO HELO REVERSE_NAME PORT LOGIN DESTADDR DESTPORT 250-ENHANCEDSTATUSCODES 250-8BITMIME 250-DSN 250 SMTPUTF8 XCLIENT ADDR=192.168.1.3 PORT=57116 LOGIN=riccardo-at-domain.local 220 postfix.domain.local ESMTP Postfix (Ubuntu) EHLO submission.domain.local 250-postfix.domain.local 250-PIPELINING ... As you can see the first EHLO is correctly done with the submission server's hostname, but the subsequent one after XCLIENT is -still- the same hostname, while I was expecting that to be the one forwarded from the MUA, or possibly a "HELO=somethingelse" in the XCLIENT line. I've looked almost everywhere and I couldn't find a way to make dovecot use the MUA's HELO string. The question I'm asking is if there is a way to have the MUA's HELO forwarded to the relay server in some way, or if this is a bug or an expected behaviour. Thanks for any help you could give -- Best regards, Riccardo Alfieri Spamhaus Technology https://www.spamhaustech.com/
Re: auth-client via SSL?
Hi, I'm resending this message, still hoping for an answer. Hello, does dovecot support tls-on-connect for AF INET based auth-client sockets? Rationale behind my question: Exim can use the Dovecot auth-client socket to delegate the SMTP-AUTH authentication to Dovecot. Currently Exim supports the AF UNIX only for this socket. Jeremy makes progress in extending this to use AF INET sockets too. While it works with clear text communication already, during testing I was to setup the auch-client socket as an TLS server (tls-on-connect). It doesn't seem to work as I'd expect. The socket still offers clear-text only. Here my configuration snippets regarding this socket ssl = yes ssl_cert = signature.asc Description: PGP signature
Re: Strategy for fts
Am 04.02.20 um 11:46 schrieb Francis Augusto Medeiros-Logeay: Hi Philon, Thanks a lot for your thoughts! Can I ask you if using Solr improved things for you? I have a mailbox with 15 years of e-mail and searching things take a long time. Here, SOLR itself searches a quarter million mails in split seconds and returns very good results. That is on a low memory average machine. If you dont mind the standard, you can change the schema, so headers (from, to) get indexed in body text. That can help narrowing results. Only problem is search through e.g. nested folders from IMAP: something like ESEARCH would be nice - https://tools.ietf.org/html/rfc6237 Peter On 04.02.2020 09:39, Philon wrote: Hi Francis, next to fts-solr there was fts-lucene. But that Lucene there seems heavily outdated why the Dovecot docs also suggest using Solr. Elasticsearch probably is similar to Solr but the later is maintained by Dovecot team. I started with downloading the Solr binary distribution to Debian with JRE preinstalled and things were running like after 10 min. Yes it’s a bit more complicated to find the schema and edit things like header size (in tips section). It’s running quite nicely since then and has zero maintenance. I will try again - I kept getting some weird errors, so I don't know if that's why I wasn't seing much of improvement. As FTS indexes are separate in external Solr instance I’d guess that it won’t interfere with dsync. What I don’t know is if dsync’ing would trigger indexing. This brings me to wonder how one could actually replicate the Solr instance!? Good question. But what I thought about doing was to install FTS on my backup instance, and if things go fine, then I install an FTS instance on my production server - that is, if one doesn't interfere with the other. I will give Solr another shot - my worries are mostly if Solr is supported on ARM (my prod instance is running on ARM) - I know Elasticsearch has an ARM build. Ii thought about the Xapian engine, but since it requires dovecot 2.3, I will have to wait. Best, Francis Philon On 31 Jan 2020, at 17:24, Francis Augusto Medeiros-Logeay wrote: Hi there, I got successfully to replicate my mail server to another dovecot install using dsync, mainly for redundancy, and it works great. I want to try to install fts, as some of the mailboxes have tens of thousands of messages, and it takes minutes to get some results when searching via IMAP on a Roundcube interface. I want to experiment with fts-solr first, and firstly on my redundant server, ie., not on my main dovecot install. Is it ok to do this? I ask because I am afraid of how this whole reindexing on the redundant install will affect the production server. Also, any tips on something else than fts-solr? I tried it once, but it was so hard to get it right, so many configurations, java, etc., that I'd rather try something else. I also could try fts-elastic or something like that, but, again, having to maintain an elasticsearch install might use more resources than I think is worth. Any thoughts on that? Best, -- Francis
Re: Strategy for fts and Replication
On February 4, 2020 11:46:31 AM GMT+01:00, Francis Augusto Medeiros-Logeay wrote: >Hi Philon, > >Thanks a lot for your thoughts! > >Can I ask you if using Solr improved things for you? I have a mailbox >with 15 years of e-mail and searching things take a long time. It a vast improvement, more or less instant results. >On 04.02.2020 09:39, Philon wrote: >> Hi Francis, >> >> next to fts-solr there was fts-lucene. But that Lucene there seems >> heavily outdated why the Dovecot docs also suggest using Solr. >> Elasticsearch probably is similar to Solr but the later is maintained >> by Dovecot team. >> >> I started with downloading the Solr binary distribution to Debian >with >> JRE preinstalled and things were running like after 10 min. Yes it’s >a >> bit more complicated to find the schema and edit things like header >> size (in tips section). It’s running quite nicely since then and has >> zero maintenance. > >I will try again - I kept getting some weird errors, so I don't know if > >that's why I wasn't seing much of improvement. >> >> As FTS indexes are separate in external Solr instance I’d guess that >> it won’t interfere with dsync. What I don’t know is if dsync’ing >would >> trigger indexing. This brings me to wonder how one could actually >> replicate the Solr instance!? > >Good question. But what I thought about doing was to install FTS on my >backup instance, and if things go fine, then I install an FTS instance >on my production server - that is, if one doesn't interfere with the >other. > >I will give Solr another shot - my worries are mostly if Solr is >supported on ARM (my prod instance is running on ARM) - I know >Elasticsearch has an ARM build. > >Ii thought about the Xapian engine, but since it requires dovecot 2.3, >I >will have to wait. > >Best, > >Francis > > >> >> Philon >> >>> On 31 Jan 2020, at 17:24, Francis Augusto Medeiros-Logeay >>> wrote: >>> >>> Hi there, >>> >>> I got successfully to replicate my mail server to another dovecot >>> install using dsync, mainly for redundancy, and it works great. >>> >>> I want to try to install fts, as some of the mailboxes have tens of >>> thousands of messages, and it takes minutes to get some results when > >>> searching via IMAP on a Roundcube interface. >>> >>> I want to experiment with fts-solr first, and firstly on my >redundant >>> server, ie., not on my main dovecot install. Is it ok to do this? I >>> ask because I am afraid of how this whole reindexing on the >redundant >>> install will affect the production server. >>> >>> Also, any tips on something else than fts-solr? I tried it once, but > >>> it was so hard to get it right, so many configurations, java, etc., >>> that I'd rather try something else. I also could try fts-elastic or >>> something like that, but, again, having to maintain an elasticsearch > >>> install might use more resources than I think is worth. Any thoughts > >>> on that? >>> >>> Best, >>> >>> -- >>> Francis >>> -- Christian Kivalo
Re: Strategy for fts and Replication
Hi Philon, Thanks a lot for your thoughts! Can I ask you if using Solr improved things for you? I have a mailbox with 15 years of e-mail and searching things take a long time. On 04.02.2020 09:39, Philon wrote: Hi Francis, next to fts-solr there was fts-lucene. But that Lucene there seems heavily outdated why the Dovecot docs also suggest using Solr. Elasticsearch probably is similar to Solr but the later is maintained by Dovecot team. I started with downloading the Solr binary distribution to Debian with JRE preinstalled and things were running like after 10 min. Yes it’s a bit more complicated to find the schema and edit things like header size (in tips section). It’s running quite nicely since then and has zero maintenance. I will try again - I kept getting some weird errors, so I don't know if that's why I wasn't seing much of improvement. As FTS indexes are separate in external Solr instance I’d guess that it won’t interfere with dsync. What I don’t know is if dsync’ing would trigger indexing. This brings me to wonder how one could actually replicate the Solr instance!? Good question. But what I thought about doing was to install FTS on my backup instance, and if things go fine, then I install an FTS instance on my production server - that is, if one doesn't interfere with the other. I will give Solr another shot - my worries are mostly if Solr is supported on ARM (my prod instance is running on ARM) - I know Elasticsearch has an ARM build. Ii thought about the Xapian engine, but since it requires dovecot 2.3, I will have to wait. Best, Francis Philon On 31 Jan 2020, at 17:24, Francis Augusto Medeiros-Logeay wrote: Hi there, I got successfully to replicate my mail server to another dovecot install using dsync, mainly for redundancy, and it works great. I want to try to install fts, as some of the mailboxes have tens of thousands of messages, and it takes minutes to get some results when searching via IMAP on a Roundcube interface. I want to experiment with fts-solr first, and firstly on my redundant server, ie., not on my main dovecot install. Is it ok to do this? I ask because I am afraid of how this whole reindexing on the redundant install will affect the production server. Also, any tips on something else than fts-solr? I tried it once, but it was so hard to get it right, so many configurations, java, etc., that I'd rather try something else. I also could try fts-elastic or something like that, but, again, having to maintain an elasticsearch install might use more resources than I think is worth. Any thoughts on that? Best, -- Francis 0xEE41D33F.asc Description: application/pgp-keys
Re: Strategy for fts and Replication
Hi Francis, next to fts-solr there was fts-lucene. But that Lucene there seems heavily outdated why the Dovecot docs also suggest using Solr. Elasticsearch probably is similar to Solr but the later is maintained by Dovecot team. I started with downloading the Solr binary distribution to Debian with JRE preinstalled and things were running like after 10 min. Yes it’s a bit more complicated to find the schema and edit things like header size (in tips section). It’s running quite nicely since then and has zero maintenance. As FTS indexes are separate in external Solr instance I’d guess that it won’t interfere with dsync. What I don’t know is if dsync’ing would trigger indexing. This brings me to wonder how one could actually replicate the Solr instance!? Philon > On 31 Jan 2020, at 17:24, Francis Augusto Medeiros-Logeay > wrote: > > Hi there, > > I got successfully to replicate my mail server to another dovecot install > using dsync, mainly for redundancy, and it works great. > > I want to try to install fts, as some of the mailboxes have tens of thousands > of messages, and it takes minutes to get some results when searching via IMAP > on a Roundcube interface. > > I want to experiment with fts-solr first, and firstly on my redundant server, > ie., not on my main dovecot install. Is it ok to do this? I ask because I am > afraid of how this whole reindexing on the redundant install will affect the > production server. > > Also, any tips on something else than fts-solr? I tried it once, but it was > so hard to get it right, so many configurations, java, etc., that I'd rather > try something else. I also could try fts-elastic or something like that, but, > again, having to maintain an elasticsearch install might use more resources > than I think is worth. Any thoughts on that? > > Best, > > -- > Francis >
