Re: Please do not remove replication
I have two postfix/(replicating)dovecot/rspamd/etc servers, 6-8 human users, on a total of 15-20 devices (iOS and macOS) Yours, Gerben Wierda (LinkedIn, Mastodon) R&A_IT_Strategy (main site) Book: Chess_and_the_Art_of_Enterprise Architecture Book: Mastering_ArchiMate YouTube_Channel On 25 Jan 2024, at 10:17, Marc wrote: How many servers do you have? How many active clients do you have? Respectfully, I would like to ask: please do not remove replication, please rethink this. Currently, replication is my life saver. I run two postfix/ dovecot combos (on different operating systems), with dovecot synchronising via replication. Both are behind a HAProxy running on the router (OPNsense), one as active, one as backup. If one of the two fails, the other takes over, and when it comes up again everything works fine and is up to date. I have had these kinds of system failures (very hard to find and turned to be hardware related) and it was the replication that made me survive the issues (even when I was far away from my systems). Mail for my small group of users (about 8) never went down, no mail message was ever lost, no manual interventions to sync were ever needed. If I want to create the same level of availability without replication, I need those two dovecots to use shared (NFS cluster) storage. But then, I have another single point of failure (NFS storage) again. So, I need two separate NFS machines that synchronise, Apart from the nightmare of making NFS secure, it means that I need to double my hardware (from two systems to four) to be protected against hardware failure (which is my goal). The replication service is the perfect small scale solution. Together with HAProxy, it enables HA in the most simple and effective way. Going the 'NFS cluster' route is not feasible for me, so if replication is removed and I am forced to upgrade, I will lose HA. So please, take small scale users like me into account. Gerben Wierda (LinkedIn <https://www.linkedin.com/in/ gerbenwierda>, Mastodon <https://newsie.social/@gctwnl>) R&A IT Strategy <https://ea.rna.nl/> (main site) Book: Chess and the Art of Enterprise Architecture Book: Mastering ArchiMate <https://ea.rna.nl/the-book- edition-iii/> YouTube Channel <http://www.youtube.com/@GerbenWierda> On 16 Jul 2023, at 18:54, Aki Tuomi via dovecot wrote: Hi! Yes, director and replicator are removed, and won't be available for pro users either. For NFS setups (or similar shared setups), we have documented a way to use Lua to run a director-like setup, see https://doc.dovecot.org/3.0/configuration_manual/ howto/director_with_lua/ Regards to replication, doveadm sync is not being removed. So you can still run doveadm sync on your system to have a primary / backup setup. Aki On 16/07/2023 18:34 EEST William Edwards via dovecot wrote: Top posting because nothing specific to reply to, sorry. Not exactly sure, but there’s another thread about the removal of Director in favour of Dovecot Pro on 3.x. Perhaps this change is related. William Edwards Op 16 jul. 2023 om 16:33 heeft Daniele het volgende geschreven: Hello, Just like Vladimir, I'm a bit concerned about this change, and I'd really appreciate if someone could let us know if the replication feature (that works so well!) will be replaced or removed; and, in case of removal, what would be recommended replacement? Thanks in advance and best regards, Daniele On 09-Jul-23 9:36 PM, Vladimir Mishonov via dovecot wrote: Hello everyone. Just saw this commit in the
Please do not remove replication
Respectfully, I would like to ask: please do not remove replication, please rethink this. Currently, replication is my life saver. I run two postfix/dovecot combos (on different operating systems), with dovecot synchronising via replication. Both are behind a HAProxy running on the router (OPNsense), one as active, one as backup. If one of the two fails, the other takes over, and when it comes up again everything works fine and is up to date. I have had these kinds of system failures (very hard to find and turned to be hardware related) and it was the replication that made me survive the issues (even when I was far away from my systems). Mail for my small group of users (about 8) never went down, no mail message was ever lost, no manual interventions to sync were ever needed. If I want to create the same level of availability without replication, I need those two dovecots to use shared (NFS cluster) storage. But then, I have another single point of failure (NFS storage) again. So, I need two separate NFS machines that synchronise, Apart from the nightmare of making NFS secure, it means that I need to double my hardware (from two systems to four) to be protected against hardware failure (which is my goal). The replication service is the perfect small scale solution. Together with HAProxy, it enables HA in the most simple and effective way. Going the 'NFS cluster' route is not feasible for me, so if replication is removed and I am forced to upgrade, I will lose HA. So please, take small scale users like me into account. Gerben Wierda (LinkedIn, Mastodon) R&A_IT_Strategy (main site) Book: Chess_and_the_Art_of_Enterprise Architecture Book: Mastering_ArchiMate YouTube_Channel On 16 Jul 2023, at 18:54, Aki Tuomi via dovecot wrote: Hi! Yes, director and replicator are removed, and won't be available for pro users either. For NFS setups (or similar shared setups), we have documented a way to use Lua to run a director-like setup, see https://doc.dovecot.org/3.0/configuration_manual/howto/ director_with_lua/ Regards to replication, doveadm sync is not being removed. So you can still run doveadm sync on your system to have a primary / backup setup. Aki On 16/07/2023 18:34 EEST William Edwards via dovecot wrote: Top posting because nothing specific to reply to, sorry. Not exactly sure, but there’s another thread about the removal of Director in favour of Dovecot Pro on 3.x. Perhaps this change is related. William Edwards Op 16 jul. 2023 om 16:33 heeft Daniele het volgende geschreven: Hello, Just like Vladimir, I'm a bit concerned about this change, and I'd really appreciate if someone could let us know if the replication feature (that works so well!) will be replaced or removed; and, in case of removal, what would be recommended replacement? Thanks in advance and best regards, Daniele On 09-Jul-23 9:36 PM, Vladimir Mishonov via dovecot wrote: Hello everyone. Just saw this commit in the official Github repo: https://github.com/dovecot/core/commit/ 4c04e4c30fd4817a8b0e11d04d9681173f696f41#diff- 5f643d8b0d1eea65d0f3c749d14d42b25a9d60f0f149bface862f5ff348412c8 Looking at the commit details, it appears that it completely removes the replication feature. I'm a bit perplexed by this change and am not sure what might be the justification for it. Personally, I find replication to be very useful, as it allows me to maintain a synchronized mirror of all of my mailboxes on my home server, for use as backup in case the primary server goes down for some reason. Perhaps there's some sort of replacement being planned for this feature? Or maybe the relevant code is simply going to be refactored to a plugin or external program, and there's nothing to worry about at all? In any case, I'd greatly appreciate if one of the developers could comment on this change. ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot- le...@dovecot.org _
Re: Replication going away?
>From what version on is replication gone? I am running 2.3.20 and it still >there. ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
I've sent a question to dovecot@dovecot.org via mail. Why is it not ending up here?
This is what my postfix server says: Nov 28 14:23:12 albus postfix/qmgr[654]: 823D71CDE2714: from=, size=9144, nrcpt=1 (que ue active) Nov 28 14:23:12 albus postfix/smtp[2294]: Untrusted TLS connection established to talvi.dovecot.org[94.237.1 05.223]:25: TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (prime256v1) server -signature RSA-PSS (2048 bits) server-digest SHA256 Nov 28 14:23:13 albus postfix/smtp[2294]: 823D71CDE2714: to=, relay=talvi.dovecot.org[9 4.237.105.223]:25, delay=2.1, delays=0.51/0.02/0.75/0.83, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as E5 C425DEEF) Nov 28 14:23:13 albus postfix/qmgr[654]: 823D71CDE2714: removed But I can't see it here nor do I get it sent back to me. ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: dovecot replication - new and cur folders on mx1 and mx2
It might have a noticeable effect on clients. I encountered (probably triggered by this in some way?) that I was unable to het the 'read' bit set in macOS Mail.app. Maybe (as I am doing HA with round robin) the Mail.app client got to one dovecot repository on one tcp connection and then on the other. Is there a reason why syncing tis move from new to cur is a bad idea? Gerben Wierda (LinkedIn <https://www.linkedin.com/in/gerbenwierda>) R&A IT Strategy <https://ea.rna.nl/> (main site) Book: Chess and the Art of Enterprise Architecture <https://ea.rna.nl/the-book/> Book: Mastering ArchiMate <https://ea.rna.nl/the-book-edition-iii/> > On 17 Jan 2023, at 22:41, Gerben Wierda wrote: > > I can confirm this in a slightly different setting, but still using two-way > sync between two dovecots. On e is 2.3.19.1 running on macOS Monterey, the > other is 2.3.20 running in an alpine container on Ubuntu. > > Gerben Wierda (LinkedIn <https://www.linkedin.com/in/gerbenwierda>) > R&A IT Strategy <https://ea.rna.nl/> (main site) > Book: Chess and the Art of Enterprise Architecture > <https://ea.rna.nl/the-book/> > Book: Mastering ArchiMate <https://ea.rna.nl/the-book-edition-iii/> > >> On 15 Jan 2023, at 23:12, Tomaz Kavcic > <mailto:tomaz.kav...@futurion.si>> wrote: >> >> Hello, >> >> I have a question in regards to specific dovecot replication behaviour and >> I'm just wondering if this is actually an expected/normal behaviour, or just >> a version issue. >> >> I'm using dovecot 2.3.16 which is packed by default with latest Ubuntu >> 22.04.1 LTS server release. I setup dovecot replication pair (mx1 - mx2) >> which is working ok. MX1 has priority 10, MX2 has priority 20. I use maildir >> (postfix + dovecot lmtp). >> >> The "strange" behaviour is this. When new mail arrives, it's by default >> delivered into "new" folder inside user directory. This email is then >> replicated to both servers (mx1 and mx2). When I login to mx1 via IMAP >> client (roundcube, outlook, etc.) that specific email is moved from "new" to >> "cur" folder on server mx1 and it's flagged also with "S", which probably >> means read flag. On server mx2, that email filename is also flagged with >> "S", but the email stays inside the "new" folder and it's not moved to >> "cur". If I want this email to be moved to "cur" on mx2 server, I have to >> login to that IMAP server as well, click on that email (which is already >> flagged as read), and after click, the email is also moved to "cur" on >> server mx2. >> >> Simply said, all new mails on mx1 server are moved to "cur" when accessed, >> but the stay in "new" folder on server mx2 until they're physically accessed >> there as well. Is this normal behaviour? >> >> I tried setup with TCP and SSH replication, and the situation is the same in >> all cases. Lastly, I tried TCPS (with SSL) as well, but that option has >> issues in 2.3.16, which is probably known already as I found multiple posts >> about these issues in this version. >> >> Thank you in advance for your answer and kind regards, Tomaz Kavcic. >> >> --- >> >> As suggested by mr. Timo Sarainen, this should be synced, so I'm posting >> doveconf -n as attachment for both servers as well. >> >> >
Re: replicator: Panic: data stack: Out of memory when allocating 268435496 bytes
> On 6 Jan 2023, at 08:53, Aki Tuomi wrote: > > > > On January 6, 2023 3:56:39 AM GMT+02:00, Gerben Wierda > wrote: >> One step further in my quest to create a replacement mail server. >> >> I now have my old mail server (2.3.19.1, macOS + MacPorts) and my new >> (2.3.20, Alpine Linux, Docker, apk package). When I turn on replication it >> works, but, after a while I see: >> >> Jan 06 00:50:31 replicator: Panic: data stack: Out of memory when allocating >> 268435496 bytes >> Jan 06 00:50:32 replicator: Fatal: master: service(replicator): child 133 >> killed with signal 6 (core dumped) >> Jan 06 00:50:32 lmtp(pid 195 user sysbh): Warning: replication(sysbh): Sync >> failure: >> Jan 06 00:50:32 lmtp(pid 195 user sysbh): Warning: replication(sysbh): >> Remote sent invalid input: - >> >> I've removed synchronous operation for now (found a message on the net >> suggesting that) but is this known and what does it mean? >> >> Gerben Wierda (LinkedIn <https://www.linkedin.com/in/gerbenwierda>) >> R&A IT Strategy <https://ea.rna.nl/> (main site) >> Book: Chess and the Art of Enterprise Architecture >> <https://ea.rna.nl/the-book/> >> Book: Mastering ArchiMate <https://ea.rna.nl/the-book-edition-iii/> >> > > Dovecot default memory limit is 256M. You should probably set > > service replicator { > vsz_limit = 2G > } > > because replicator might have to use more memory, especially for larger > indexes. > > Aki Coming back to this one. doveadm config says everything is vsz_limit = 18446744073709551615 B Isn't that more than 2G already (nd certainly more than 256M?)? I also tried putting default_vsz_limit = 2G in dovecot.conf but that doesn't change anything either. What am I missing?
Re: dovecot replication - new and cur folders on mx1 and mx2
I can confirm this in a slightly different setting, but still using two-way sync between two dovecots. On e is 2.3.19.1 running on macOS Monterey, the other is 2.3.20 running in an alpine container on Ubuntu. Gerben Wierda (LinkedIn <https://www.linkedin.com/in/gerbenwierda>) R&A IT Strategy <https://ea.rna.nl/> (main site) Book: Chess and the Art of Enterprise Architecture <https://ea.rna.nl/the-book/> Book: Mastering ArchiMate <https://ea.rna.nl/the-book-edition-iii/> > On 15 Jan 2023, at 23:12, Tomaz Kavcic wrote: > > Hello, > > I have a question in regards to specific dovecot replication behaviour and > I'm just wondering if this is actually an expected/normal behaviour, or just > a version issue. > > I'm using dovecot 2.3.16 which is packed by default with latest Ubuntu > 22.04.1 LTS server release. I setup dovecot replication pair (mx1 - mx2) > which is working ok. MX1 has priority 10, MX2 has priority 20. I use maildir > (postfix + dovecot lmtp). > > The "strange" behaviour is this. When new mail arrives, it's by default > delivered into "new" folder inside user directory. This email is then > replicated to both servers (mx1 and mx2). When I login to mx1 via IMAP client > (roundcube, outlook, etc.) that specific email is moved from "new" to "cur" > folder on server mx1 and it's flagged also with "S", which probably means > read flag. On server mx2, that email filename is also flagged with "S", but > the email stays inside the "new" folder and it's not moved to "cur". If I > want this email to be moved to "cur" on mx2 server, I have to login to that > IMAP server as well, click on that email (which is already flagged as read), > and after click, the email is also moved to "cur" on server mx2. > > Simply said, all new mails on mx1 server are moved to "cur" when accessed, > but the stay in "new" folder on server mx2 until they're physically accessed > there as well. Is this normal behaviour? > > I tried setup with TCP and SSH replication, and the situation is the same in > all cases. Lastly, I tried TCPS (with SSL) as well, but that option has > issues in 2.3.16, which is probably known already as I found multiple posts > about these issues in this version. > > Thank you in advance for your answer and kind regards, Tomaz Kavcic. > > --- > > As suggested by mr. Timo Sarainen, this should be synced, so I'm posting > doveconf -n as attachment for both servers as well. > >
Re: replicator: Panic: data stack: Out of memory when allocating 268435496 bytes
dovecot-pgsql.conf > driver = sql > > } > > protocol imap { > mail_max_userip_connections = 50 > mail_plugins = $mail_plugins notify replication > } > > protocol pop3 { > mail_max_userip_connections = 50 > mail_plugins = $mail_plugins notify replication > } > > protocol imaps { > mail_max_userip_connections = 25 > mail_plugins = $mail_plugins notify replication > } > > protocol pop3s { > mail_max_userip_connections = 25 > mail_plugins = $mail_plugins notify replication > } > > > service managesieve-login { > process_limit = 1000 > vsz_limit = 1g > inet_listener sieve { >port = 4190 > } > } > > verbose_proctitle = yes > > replication_max_conns = 100 > > replication_full_sync_interval = 1d > > service replicator { > client_limit = 0 > drop_priv_before_exec = no > idle_kill = 4294967295s > process_limit = 1 > process_min_avail = 0 > service_count = 0 > vsz_limit = 8g >unix_listener replicator-doveadm { >mode = 0600 >user = vmail > } > vsz_limit = 8192M > } > > > service aggregator { > process_limit = 1000 > #vsz_limit = 1g > fifo_listener replication-notify-fifo { >user = vmail >group = vmail >mode = 0666 > } > > } > > service pop3-login { > process_limit = 1000 > client_limit = 100 > vsz_limit = 512m > } > > > service imap-urlauth-login { > process_limit = 1000 > client_limit = 1000 > vsz_limit = 1g > } > > > service imap-login { > process_limit=1000 > client_limit = 1000 > vsz_limit = 1g > } > > > protocol sieve { > managesieve_implementation_string = Dovecot Pigeonhole > managesieve_max_line_length = 65536 > } > > > > > #Addition ssl config > !include sni.conf > > with sni cert support (examples) > > # cat sni.conf > #sni.conf > ssl = yes > verbose_ssl = yes > ssl_dh = ssl_prefer_server_ciphers = yes > #ssl_min_protocol = TLSv1.2 > > #Default *.scom.ca > ssl_key = ssl_cert = ssl_ca = > local_name .scom.ca { > ssl_key = ssl_cert = ssl_ca = > } > > local_name mail.clancyca.com { > ssl_key = ssl_cert = ssl_ca = } > > local_name mail.paulkudla.net { > ssl_key = ssl_cert = ssl_ca = } > > local_name mail.ekst.ca { > ssl_key = ssl_cert = ssl_ca = } > > local_name mail.hamletdevelopments.ca { > ssl_key = ssl_cert = ssl_ca = } > > pg sql support supporting replication > > # cat dovecot-pgsql.conf > driver = pgsql > connect = host=localhost port=5433 dbname=scom_billing user=pgsql password= > default_pass_scheme = PLAIN > > password_query = SELECT username as user, password FROM email_users WHERE > username = '%u' and password <> 'alias' and status = True and destination = > '%u' > > user_query = SELECT home, uid, gid FROM email_users WHERE username = '%u' and > password <> 'alias' and status = True and destination = '%u' > > #iterate_query = SELECT user, password FROM email_users WHERE username = '%u' > and password <> 'alias' and status = True and destination = '%u' > > iterate_query = SELECT "username" as user, domain FROM email_users WHERE > status = True and alias_flag = False > > > > > > Happy Friday !!! > Thanks - paul > > Paul Kudla > > > Scom.ca Internet Services <http://www.scom.ca> > 004-1009 Byron Street South > Whitby, Ontario - Canada > L1N 4S3 > > Toronto 416.642.7266 > Main 1.866.411.7266 > Fax 1.888.892.7266 > Email p...@scom.ca > > On 1/6/2023 5:32 AM, Gerben Wierda wrote: >>> On 6 Jan 2023, at 08:53, Aki Tuomi >> <mailto:aki.tu...@open-xchange.com>> wrote: >>> >>> >>> >>> On January 6, 2023 3:56:39 AM GMT+02:00, Gerben Wierda >>> mailto:gerben.wie...@rna.nl>> wrote: >>>> One step further in my quest to create a replacement mail server. >>>> >>>> I now have my old mail server (2.3.19.1, macOS + MacPorts) and my new >>>> (2.3.20, Alpine Linux, Docker, apk package). When I turn on replication it >>>> works, but, after a while I see: >>>> >>>> Jan 06 00:50:31 replicator: Panic: data stack: Out of memory when >>>> allocating 268435496 bytes >>>> Jan 06 00:50:32 replicator: Fatal: master: service(replicator): child 133 >>>> killed with signal 6 (core dumped) >>>> Jan 06 00:50:32 lmtp(pid 195 user sysbh): Warning: replication(sysbh): >>>> Sync failure: >>>> Jan 06 00:50:32 lmtp(pid 195 user sysbh): Warning: replication(sysbh): >>>> Remote sent invalid input: - >>>> >>>> I've removed synchronous operation for now (found a message on the net >>>> suggesting that) but is this known and what does it mean? >>>> >>>> Gerben Wierda (LinkedIn <https://www.linkedin.com/in/gerbenwierda >>>> <https://www.linkedin.com/in/gerbenwierda>>) >>>> R&A IT Strategy <https://ea.rna.nl/ <https://ea.rna.nl/>> (main site) >>>> Book: Chess and the Art of Enterprise Architecture >>>> <https://ea.rna.nl/the-book/ <https://ea.rna.nl/the-book/>> >>>> Book: Mastering ArchiMate <https://ea.rna.nl/the-book-edition-iii/ >>>> <https://ea.rna.nl/the-book-edition-iii/>> >>>> >>> >>> Dovecot default memory limit is 256M. You should probably set >>> >>> service replicator { >>> vsz_limit = 2G >>> } >>> >>> because replicator might have to use more memory, especially for larger >>> indexes. >>> >>> Aki >> That is a good tip as well. >> I had followed this bit of experience from someone else: >> https://marc.info/?l=dovecot&m=164438199727640 >> <https://marc.info/?l=dovecot&m=164438199727640>, haven't seen any err >> message since. But that might be because they are in sync now and both sides >> are aware. Can I trigger full replication again so I can test? >> Gerben >> -- >> This message has been scanned for viruses and >> dangerous content by *MailScanner* <http://www.mailscanner.info/>, and is >> believed to be clean.
Re: replicator: Panic: data stack: Out of memory when allocating 268435496 bytes
> On 6 Jan 2023, at 08:53, Aki Tuomi wrote: > > > > On January 6, 2023 3:56:39 AM GMT+02:00, Gerben Wierda > wrote: >> One step further in my quest to create a replacement mail server. >> >> I now have my old mail server (2.3.19.1, macOS + MacPorts) and my new >> (2.3.20, Alpine Linux, Docker, apk package). When I turn on replication it >> works, but, after a while I see: >> >> Jan 06 00:50:31 replicator: Panic: data stack: Out of memory when allocating >> 268435496 bytes >> Jan 06 00:50:32 replicator: Fatal: master: service(replicator): child 133 >> killed with signal 6 (core dumped) >> Jan 06 00:50:32 lmtp(pid 195 user sysbh): Warning: replication(sysbh): Sync >> failure: >> Jan 06 00:50:32 lmtp(pid 195 user sysbh): Warning: replication(sysbh): >> Remote sent invalid input: - >> >> I've removed synchronous operation for now (found a message on the net >> suggesting that) but is this known and what does it mean? >> >> Gerben Wierda (LinkedIn <https://www.linkedin.com/in/gerbenwierda>) >> R&A IT Strategy <https://ea.rna.nl/> (main site) >> Book: Chess and the Art of Enterprise Architecture >> <https://ea.rna.nl/the-book/> >> Book: Mastering ArchiMate <https://ea.rna.nl/the-book-edition-iii/> >> > > Dovecot default memory limit is 256M. You should probably set > > service replicator { > vsz_limit = 2G > } > > because replicator might have to use more memory, especially for larger > indexes. > > Aki That is a good tip as well. I had followed this bit of experience from someone else: https://marc.info/?l=dovecot&m=164438199727640 <https://marc.info/?l=dovecot&m=164438199727640>, haven't seen any err message since. But that might be because they are in sync now and both sides are aware. Can I trigger full replication again so I can test? Gerben
replicator: Panic: data stack: Out of memory when allocating 268435496 bytes
One step further in my quest to create a replacement mail server. I now have my old mail server (2.3.19.1, macOS + MacPorts) and my new (2.3.20, Alpine Linux, Docker, apk package). When I turn on replication it works, but, after a while I see: Jan 06 00:50:31 replicator: Panic: data stack: Out of memory when allocating 268435496 bytes Jan 06 00:50:32 replicator: Fatal: master: service(replicator): child 133 killed with signal 6 (core dumped) Jan 06 00:50:32 lmtp(pid 195 user sysbh): Warning: replication(sysbh): Sync failure: Jan 06 00:50:32 lmtp(pid 195 user sysbh): Warning: replication(sysbh): Remote sent invalid input: - I've removed synchronous operation for now (found a message on the net suggesting that) but is this known and what does it mean? Gerben Wierda (LinkedIn <https://www.linkedin.com/in/gerbenwierda>) R&A IT Strategy <https://ea.rna.nl/> (main site) Book: Chess and the Art of Enterprise Architecture <https://ea.rna.nl/the-book/> Book: Mastering ArchiMate <https://ea.rna.nl/the-book-edition-iii/>
Re: postfix connects to dovecot lmtp socket, but nothing is delivered
Hmm: according to the documentation, the default is "imap pop3 lmtp" But my installation (2.3.20, Alpine apk) for some reason only ran "imap" by default G > On 5 Jan 2023, at 21:27, Gerben Wierda wrote: > > Boy do I feel stupid. > > protocols was not set in the new version's dovecot.conf. And default it only > contains imap. My old server had an override in dovecot.conf and I copied > conf.d over but started with afresh dovecot.conf, so lmtp was missing. > > G > >> On 5 Jan 2023, at 18:39, Gerben Wierda > <mailto:gerben.wie...@rna.nl>> wrote: >> >>> On 5 Jan 2023, at 18:17, dove...@ptld.com <mailto:dove...@ptld.com> wrote: >>> >>>> service lmtp { >>>> inet_listener lmtp { >>>> address = 127.0.0.1 >>>> port = 24 >>>> } >>> >>>> unix_listener /var/spool/postfix/private/lmtp { >>>> mode = 0660 >>>> user = postfix >>>> group = postfix >>>> } >>> >>> >>> Looks like you are activating both a socket and a port. Im not sure what >>> dovecot does, does it bind to both or does it pick one and ignore the >>> other. You only need to use one or the other. >>> >>> And what does your postfix side look like? Where is postfix trying to >>> connect? To the socket or the port? For using a socket postfix would look >>> something like: >>> >>>virtual_transport = lmtp:unix:private/lmtp >>> >> >> I removed the tcp port for now. >> >> / # doveconf service/lmtp >> service lmtp { >> chroot = >> client_limit = 1 >> drop_priv_before_exec = no >> executable = lmtp >> extra_groups = $default_internal_group >> group = >> idle_kill = 0 >> privileged_group = >> process_limit = 0 >> process_min_avail = 0 >> protocol = lmtp >> service_count = 0 >> type = >> unix_listener /var/spool/postfix/private/dovecot-lmtp { >> group = mail >> mode = 0660 >> user = postfix >> } >> unix_listener lmtp { >> group = >> mode = 0666 >> user = >> } >> user = >> vsz_limit = 18446744073709551615 B >> } >> >> postfix config: >> >> mailbox_transport = lmtp:unix:private/dovecot-lmtp >> virtual_transport = lmtp:unix:private/dovecot-lmtp >> >> I had confused postfix's own lmtp socket with dovecot's, which was >> originally named dovecot-lmtp but when it wasn't created I though my config >> was wrong. >> >> What seems to happen is that /var/spool/postfix/private/dovecot-lmtp isn't >> created by dovecot at launch, but /var/spool/postfix/private/auth is >> >> G >
Re: postfix connects to dovecot lmtp socket, but nothing is delivered
Boy do I feel stupid. protocols was not set in the new version's dovecot.conf. And default it only contains imap. My old server had an override in dovecot.conf and I copied conf.d over but started with afresh dovecot.conf, so lmtp was missing. G > On 5 Jan 2023, at 18:39, Gerben Wierda wrote: > >> On 5 Jan 2023, at 18:17, dove...@ptld.com <mailto:dove...@ptld.com> wrote: >> >>> service lmtp { >>> inet_listener lmtp { >>> address = 127.0.0.1 >>> port = 24 >>> } >> >>> unix_listener /var/spool/postfix/private/lmtp { >>> mode = 0660 >>> user = postfix >>> group = postfix >>> } >> >> >> Looks like you are activating both a socket and a port. Im not sure what >> dovecot does, does it bind to both or does it pick one and ignore the other. >> You only need to use one or the other. >> >> And what does your postfix side look like? Where is postfix trying to >> connect? To the socket or the port? For using a socket postfix would look >> something like: >> >>virtual_transport = lmtp:unix:private/lmtp >> > > I removed the tcp port for now. > > / # doveconf service/lmtp > service lmtp { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = lmtp > extra_groups = $default_internal_group > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = lmtp > service_count = 0 > type = > unix_listener /var/spool/postfix/private/dovecot-lmtp { > group = mail > mode = 0660 > user = postfix > } > unix_listener lmtp { > group = > mode = 0666 > user = > } > user = > vsz_limit = 18446744073709551615 B > } > > postfix config: > > mailbox_transport = lmtp:unix:private/dovecot-lmtp > virtual_transport = lmtp:unix:private/dovecot-lmtp > > I had confused postfix's own lmtp socket with dovecot's, which was originally > named dovecot-lmtp but when it wasn't created I though my config was wrong. > > What seems to happen is that /var/spool/postfix/private/dovecot-lmtp isn't > created by dovecot at launch, but /var/spool/postfix/private/auth is > > G
Re: postfix connects to dovecot lmtp socket, but nothing is delivered
> On 5 Jan 2023, at 18:17, dove...@ptld.com wrote: > >> service lmtp { >> inet_listener lmtp { >> address = 127.0.0.1 >> port = 24 >> } > >> unix_listener /var/spool/postfix/private/lmtp { >> mode = 0660 >> user = postfix >> group = postfix >> } > > > Looks like you are activating both a socket and a port. Im not sure what > dovecot does, does it bind to both or does it pick one and ignore the other. > You only need to use one or the other. > > And what does your postfix side look like? Where is postfix trying to > connect? To the socket or the port? For using a socket postfix would look > something like: > >virtual_transport = lmtp:unix:private/lmtp > I removed the tcp port for now. / # doveconf service/lmtp service lmtp { chroot = client_limit = 1 drop_priv_before_exec = no executable = lmtp extra_groups = $default_internal_group group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = lmtp service_count = 0 type = unix_listener /var/spool/postfix/private/dovecot-lmtp { group = mail mode = 0660 user = postfix } unix_listener lmtp { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } postfix config: mailbox_transport = lmtp:unix:private/dovecot-lmtp virtual_transport = lmtp:unix:private/dovecot-lmtp I had confused postfix's own lmtp socket with dovecot's, which was originally named dovecot-lmtp but when it wasn't created I though my config was wrong. What seems to happen is that /var/spool/postfix/private/dovecot-lmtp isn't created by dovecot at launch, but /var/spool/postfix/private/auth is G
Re: postfix connects to dovecot lmtp socket, but nothing is delivered
> On 5 Jan 2023, at 18:17, dove...@ptld.com wrote: > >> service lmtp { >> inet_listener lmtp { >> address = 127.0.0.1 >> port = 24 >> } > >> unix_listener /var/spool/postfix/private/lmtp { >> mode = 0660 >> user = postfix >> group = postfix >> } > > > Looks like you are activating both a socket and a port. Im not sure what > dovecot does, does it bind to both or does it pick one and ignore the other. > You only need to use one or the other. > > And what does your postfix side look like? Where is postfix trying to > connect? To the socket or the port? For using a socket postfix would look > something like: > >virtual_transport = lmtp:unix:private/lmtp Ha, my mistake. As dovecot had not created the socket I assumed th elmtp socket was the one the new config had created, so I changed the config to use that one. Now I only have to find out why dovecot doesn't create the /var/spool/postfix/private/dovecot-lmtp socket The inet_listener was added to see if I could work around this. G
postfix connects to dovecot lmtp socket, but nothing is delivered
I am setting up a new server combination (postfix + dovecot). dovecot's auth service works for postfix. Just lmtp does not and I need to find out why. I don't see anything in the dovecot logging, but I see this on the postfix side (debug) Jan 05 16:11:59 snape postfix/lmtp[127]: connection established Jan 05 16:11:59 snape postfix/lmtp[127]: master_notify: status 0 Jan 05 16:11:59 snape postfix/lmtp[127]: deliver_request_initial: send initial response Jan 05 16:11:59 snape postfix/lmtp[127]: send attr protocol = delivery_request_protocol Jan 05 16:16:59 snape postfix/lmtp[126]: smtp_get: timeout Jan 05 16:16:59 snape postfix/lmtp[126]: connect to subsystem private/defer Jan 05 16:16:59 snape postfix/lmtp[126]: private/defer socket: wanted attribute: protocol Jan 05 16:16:59 snape postfix/lmtp[126]: input attribute name: protocol Jan 05 16:16:59 snape postfix/lmtp[126]: input attribute value: delivery_status_protocol Jan 05 16:16:59 snape postfix/lmtp[126]: private/defer socket: wanted attribute: (list terminator) Jan 05 16:16:59 snape postfix/lmtp[126]: input attribute name: (end) Jan 05 16:16:59 snape postfix/lmtp[126]: send attr nrequest = 0 Jan 05 16:16:59 snape postfix/lmtp[126]: send attr flags = 0 Jan 05 16:16:59 snape postfix/lmtp[126]: send attr queue_id = C71B3D1262 Jan 05 16:16:59 snape postfix/lmtp[126]: send attr original_recipient = sy...@rna.nl Jan 05 16:16:59 snape postfix/lmtp[126]: send attr recipient = sy...@rna.nl Jan 05 16:16:59 snape postfix/lmtp[126]: send attr offset = 18446744073709551615 Jan 05 16:16:59 snape postfix/lmtp[126]: send attr dsn_orig_rcpt = rfc822;sy...@rna.nl Jan 05 16:16:59 snape postfix/lmtp[126]: send attr notify_flags = 0 Jan 05 16:16:59 snape postfix/lmtp[126]: send attr status = 4.4.2 Jan 05 16:16:59 snape postfix/lmtp[126]: send attr diag_type = Jan 05 16:16:59 snape postfix/lmtp[126]: send attr diag_text = Jan 05 16:16:59 snape postfix/lmtp[126]: send attr mta_type = Jan 05 16:16:59 snape postfix/lmtp[126]: send attr mta_mname = Jan 05 16:16:59 snape postfix/lmtp[126]: send attr action = delayed Jan 05 16:16:59 snape postfix/lmtp[126]: send attr reason = conversation with snape.rna.nl[private/lmtp] timed out while receiving the initial server greeting Jan 05 16:16:59 snape postfix/lmtp[126]: private/defer socket: wanted attribute: status Jan 05 16:16:59 snape postfix/lmtp[126]: input attribute name: status Jan 05 16:16:59 snape postfix/lmtp[126]: input attribute value: 0 Jan 05 16:16:59 snape postfix/lmtp[126]: private/defer socket: wanted attribute: (list terminator) Jan 05 16:16:59 snape postfix/lmtp[126]: input attribute name: (end) Jan 05 16:16:59 snape postfix/lmtp[126]: C71B3D1262: to=, relay=snape.rna.nl[private/lmtp], delay=300, delays=0.02/0/300/0, dsn=4.4.2, status=deferred (conversation with snape.rna.nl[private/lmtp] timed out while receiving the initial server greeting) Jan 05 16:16:59 snape postfix/lmtp[126]: flush_add: site rna.nl id C71B3D1262 Jan 05 16:16:59 snape postfix/lmtp[126]: match_list_match: rna.nl: no match Jan 05 16:16:59 snape postfix/lmtp[126]: flush_add: site rna.nl id C71B3D1262 status 4 Jan 05 16:16:59 snape postfix/lmtp[126]: deliver_request_final: send: "conversation with snape.rna.nl[private/lmtp] timed out while receiving the initial server greeting" -1 Jan 05 16:16:59 snape postfix/lmtp[126]: send attr status = 4.4.2 Jan 05 16:16:59 snape postfix/lmtp[126]: send attr diag_type = Jan 05 16:16:59 snape postfix/lmtp[126]: send attr diag_text = Jan 05 16:16:59 snape postfix/lmtp[126]: send attr mta_type = Jan 05 16:16:59 snape postfix/lmtp[127]: master_notify: status 1 Jan 05 16:16:59 snape postfix/lmtp[126]: send attr mta_mname = Jan 05 16:16:59 snape postfix/lmtp[127]: connection closed Jan 05 16:16:59 snape postfix/lmtp[126]: send attr action = Jan 05 16:16:59 snape postfix/lmtp[126]: send attr reason = conversation with snape.rna.nl[private/lmtp] timed out while receiving the initial server greeting Jan 05 16:16:59 snape postfix/lmtp[126]: send attr status = 4294967295 Jan 05 16:16:59 snape postfix/lmtp[126]: master_notify: status 1 Jan 05 16:16:59 snape postfix/lmtp[126]: connection closed It seems dovecot doesn't react to whatever postfix is doing on the socket. Another possibility is that the socket doesn't really work and dovecot never gets a connection request. How can I debug this? Might I test if the socket is the problem by running lmtp on the dovecot side on TCP on 127.0.0.1 and have postfix use that? Just to see if it works? If so, how do I tell postfix to do that? In dovecot I have added the tcp: service lmtp { inet_listener lmtp { address = 127.0.0.1 port = 24 } unix_listener /var/spool/postfix/private/lmtp { mode = 0660 user = postfix group = postfix } } service lmtp { executable = lmtp -L } protocol lmtp { info_log_path = /var/log/mail/dovecot-lmtp.log } Any other tips? Gerben Wierda (LinkedIn <https:/
Up to date dovecot packages on Ubuntu 22.4LTS
I just found out that Ubuntu Linux 22.04 LTS has a dovecot apt package that is 2.3.16. Debian 11 (stable) is at dovecot 2.3.13 with apt. if I looked correctly. Dovecot itself is at 2.3.20. That surprised me (I am new to the Linux distro and package management world) as Ubuntu 22.04 is a Long-Time Support version, and I was expecting packages to be updated as well (as many users will want updated packages. How does that work in the dovecot world? How come macOS MacPorts (2.3.19) has more recent packages than any of the large Debian-based Linuxes? Gerben Wierda (LinkedIn <https://www.linkedin.com/in/gerbenwierda>) R&A IT Strategy <https://ea.rna.nl/> (main site) Book: Chess and the Art of Enterprise Architecture <https://ea.rna.nl/the-book/> Book: Mastering ArchiMate <https://ea.rna.nl/the-book-edition-iii/>
Re: Migrating, syncing, maybe load-balancing/failover two dovecot servers?
ylib 0x0001050f52e6 io_loop_run + 81 -> 18 libdovecot.0.dylib 0x00010506d5e0 master_service_run + 24 -> 19 doveadm-server 0x0001048aac3f main + 292 -> 20 dyld 0x00011487652e start + 462 Jan 04 20:17:05 doveadm(74522): Fatal: master: service(doveadm): child 74522 killed with signal 6 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps) Turns out, this is a known (and pretty old) problem (https://www.mail-archive.com/dovecot%40dovecot.org/msg85388.html) and my dovecot on the old server (macOS + MacPorts) is newer than the dovecot on the new one. I should go back to a 2.3.16 on the old server. It seems the syncing works (or has worked) nonetheless, but it doesn't feel good. Gerben Wierda (LinkedIn <https://www.linkedin.com/in/gerbenwierda>) R&A IT Strategy <https://ea.rna.nl/> (main site) Book: Chess and the Art of Enterprise Architecture <https://ea.rna.nl/the-book/> Book: Mastering ArchiMate <https://ea.rna.nl/the-book-edition-iii/> > On 4 Jan 2023, at 13:54, Paul Kudla wrote: > > > maybe look a replicator / replication > > its designed to do exactly that > > > > > Happy Wednesday !!! > Thanks - paul > > Paul Kudla > > > Scom.ca Internet Services <http://www.scom.ca> > 004-1009 Byron Street South > Whitby, Ontario - Canada > L1N 4S3 > > Toronto 416.642.7266 > Main 1.866.411.7266 > Fax 1.888.892.7266 > Email p...@scom.ca > > On 1/4/2023 7:46 AM, Gerben Wierda wrote: >> I am in the process of migrating from dovecot on one OS (macOS/darwin) to a >> new server running dovecot with another OS (Ubuntu Linux 22.4). >> I have mostly copied/adapted the setup of the old server to the new. I am in >> the process of finishing that and adding some stuff that still needs to be >> added/migrated, like rspamd. And the data of course before the new one takes >> over from the old. >> I have done a migration before (MacOS X Server dovecot to MacPorts dovecot >> on macOS), many years ago, I recall that I used dovecot syncing but also >> rsync and I don't really recall (and anyway, the software has changed since) >> I have been thinking about keeping them both alive, with one as a failover >> for the other. They will not share their storage (e.g. NFS), So, I was >> wondering if I can do something with syncing between instances and dovecot >> director. I have been looking at the documentation, but a quick scan reveals >> I cannot locate some sort of tutorial and I am uncertain what will work and >> what not. >> If keeping both alive in parallel is too problematic, it is OK to have >> regular syncing in one direction (old to new) at first and then switch over >> and have syncing in the other direction (new to old) >> Can someone enlighten me? >> Gerben Wierda (LinkedIn <https://www.linkedin.com/in/gerbenwierda>) >> R&A IT Strategy <https://ea.rna.nl/> (main site) >> Book: Chess and the Art of Enterprise Architecture >> <https://ea.rna.nl/the-book/> >> Book: Mastering ArchiMate <https://ea.rna.nl/the-book-edition-iii/> >> -- >> This message has been scanned for viruses and >> dangerous content by *MailScanner* <http://www.mailscanner.info/>, and is >> believed to be clean.
CRAM MD5 passwd db permission issue
I am busy migrating. I am moving from macOS+MacPorts to Ubuntu+Docker On the old system, I have this in the dovecot config: mail_uid = _dovecot mail_gid = mail mail_privileged_group = mail mail_access_groups = mail This seems weird to me, I think the dovecot user should be in group dovecot only if I understand the docs. On the old system dovecot, postfix, dovenull and rspamd are all members of the mail group. On that system, the cram md5 passwd database (file) has these permissions: drwxr-xr-x 3 root wheel96 Feb 2 2021 . drwxr-xr-x 22 root admin 704 Jan 4 15:17 .. -rw-r- 1 root mail 1234 Feb 2 2021 cram-md5.pwd and that has worked like that for many years, basically starting with Mac OS X Server, surviving all kinds of macOS migrations. On my new Ubuntu system I've copied this setup over: drwxr-xr-x 2 root root 4096 Jan 4 09:49 . drwxr-xr-x 7 root root 4096 Jan 4 15:21 .. -rw-r- 1 root mail 1234 Feb 2 2021 cram-md5.pwd mail_uid = dovecot mail_gid = mail mail_privileged_group = mail mail_access_groups = mail But: Jan 04 15:40:08 auth: Error: passwd-file /etc/dovecot/etc/cram-md5.pwd:open(/etc/dovecot/etc/cram-md5.pwd) failed: Permission denied (euid=91(dovecot) egid=91(dovecot) missing +r perm: /etc/dovecot/etc/cram-md5.pwd, we're not in group 8(mail), dir owned by 0:0 mode=0755) And really, dovecot is in group mail. From /etc/group: mail:x:8:postfix,dovecot dovenull:x:90: dovecot:x:91: And from /etc/passwd: dovenull:x:90:90::/home/dovenull:/usr/sbin/nologin dovecot:x:91:91::/home/dovecot:/usr/sbin/nologin So, that I get this error baffles me. Gerben Wierda (LinkedIn <https://www.linkedin.com/in/gerbenwierda>) R&A IT Strategy <https://ea.rna.nl/> (main site) Book: Chess and the Art of Enterprise Architecture <https://ea.rna.nl/the-book/> Book: Mastering ArchiMate <https://ea.rna.nl/the-book-edition-iii/>
Migrating, syncing, maybe load-balancing/failover two dovecot servers?
I am in the process of migrating from dovecot on one OS (macOS/darwin) to a new server running dovecot with another OS (Ubuntu Linux 22.4). I have mostly copied/adapted the setup of the old server to the new. I am in the process of finishing that and adding some stuff that still needs to be added/migrated, like rspamd. And the data of course before the new one takes over from the old. I have done a migration before (MacOS X Server dovecot to MacPorts dovecot on macOS), many years ago, I recall that I used dovecot syncing but also rsync and I don't really recall (and anyway, the software has changed since) I have been thinking about keeping them both alive, with one as a failover for the other. They will not share their storage (e.g. NFS), So, I was wondering if I can do something with syncing between instances and dovecot director. I have been looking at the documentation, but a quick scan reveals I cannot locate some sort of tutorial and I am uncertain what will work and what not. If keeping both alive in parallel is too problematic, it is OK to have regular syncing in one direction (old to new) at first and then switch over and have syncing in the other direction (new to old) Can someone enlighten me? Gerben Wierda (LinkedIn <https://www.linkedin.com/in/gerbenwierda>) R&A IT Strategy <https://ea.rna.nl/> (main site) Book: Chess and the Art of Enterprise Architecture <https://ea.rna.nl/the-book/> Book: Mastering ArchiMate <https://ea.rna.nl/the-book-edition-iii/>
macOS ManageSieve client?
I have a dovecot & dovecot-sieve running under MacPorts on my macOS ’server’. I do a very low tech maintenance on sieve, by using the cli on the server and edit sieve by hand for myself. I’d like to open this up to other users. How do people use this from their macOS clients? For this, the ManageSieve protocol exists and this is implemented by dovecot-sieve, but other than installing roundcube and offering a web-based mail client that also supports ManageSieve, is there another way? A ManageSieve client that directly runs on the macOS client and interfaces with dovecot-sieve on the server? After all, routing mail to different mailboxes better happens server-side so that all clients profit. Gerben Wierda (LinkedIn <https://www.linkedin.com/in/gerbenwierda>) R&A IT Strategy <https://ea.rna.nl/> (main site) Book: Chess and the Art of Enterprise Architecture <https://ea.rna.nl/the-book/> Book: Mastering ArchiMate <https://ea.rna.nl/the-book-edition-iii/>
Re: Is it possible to enable push notifications for iOS/macOS mail clients on a stock dovecot?
This requires the ongoing use of macOS Server (as does the patch that is part of the MacPorts port of dovecot2). The issue then is: how do you keep getting certificates for push mail if mail services have been removed from macOS Server? I can configure this today but on Oct 20, my apns certificate expires. Then what? Gerben Wierda Chess and the Art of Enterprise Architecture Mastering ArchiMate Architecture for Real Enterprises at InfoWorld On Slippery Ice at EAPJ > On 9 Jan 2020, at 00:19, Steffan Cline wrote: > > https://github.com/st3fan/dovecot-xaps-daemon > > Works PERFECTLY. I use it on CentOS 6 and 7. > > > Thanks, > Steffan Cline > stef...@hldns.com > 602-793-0014 > >>> On Jan 8, 2020, at 2:39 PM, infoomatic wrote: >>> >> I have no idea about that Mac proprietary stuff ... however, >> https://doc.dovecot.org/configuration_manual/push_notification/ might be >> a solution/workaround for you. >> >> regards, >> >> infoomatic >> >> >>> Am 08.01.20 um 22:33 schrieb Gerben Wierda: >>> Now that Apple has abandoned mail server capabilities in macOS Server.app >>> we have to move to an open source solution. I have done this using Macports >>> and it works well (no thanks to Apple as it never published the in 2018 >>> promised migration info for mail services). >>> >>> The MacPorts Apple Push Notification variant based on a patch written a few >>> years ago for dovecot supports APNS but it requires you to export a >>> certificate form your old High Sierra Server.app. With the patch and that >>> certificate, dovecot tells an email client it supports XAPPLEPUSHSERVICE >>> and that makes Apple mail clients think they are talking to macOS Server >>> and they switch to push notifications for updates. Now, that certificate >>> needs to be renewed yearly by Server.app and imported by hand etc. At some >>> point this will stop working (apart from that you need to keep an old macOS >>> Server around to renew) >>> >>> Is there another way to get this working with an open source dovecot, >>> outside of using Apple’s push notification service? Some other standard? >>> Some other service? Probably not, right? >>> >>> G >>
Is it possible to enable push notifications for iOS/macOS mail clients on a stock dovecot?
Now that Apple has abandoned mail server capabilities in macOS Server.app we have to move to an open source solution. I have done this using Macports and it works well (no thanks to Apple as it never published the in 2018 promised migration info for mail services). The MacPorts Apple Push Notification variant based on a patch written a few years ago for dovecot supports APNS but it requires you to export a certificate form your old High Sierra Server.app. With the patch and that certificate, dovecot tells an email client it supports XAPPLEPUSHSERVICE and that makes Apple mail clients think they are talking to macOS Server and they switch to push notifications for updates. Now, that certificate needs to be renewed yearly by Server.app and imported by hand etc. At some point this will stop working (apart from that you need to keep an old macOS Server around to renew) Is there another way to get this working with an open source dovecot, outside of using Apple’s push notification service? Some other standard? Some other service? Probably not, right? G
Re: iOS Client Not Showing Email Previews, Dovecot Issue?
> On 2 Jan 2020, at 01:18, a...@globalchangemusic.org wrote: > > Greetings, > > Just wondering if anybody would know why one of our users' iOS Mail app > client doesn't show the email previews (the first few sentences of the email)? > > We had migrated from an earlier (2.2.4) version of Dovecot to the latest > version and once that was done, the user began to report this issue. I don't > know if it's a Dovecot issue or not, but can anyone point me in the right > direction to begin to debug? I am running the latest MacPorts dovecot and latest iOS and see the first line. I have run into some issues with Mail.app on macOS that were solved by removing the account and recreating again. macOS Mail.app kept settings around on how to connect to IMAP (from years of upgrading an existing dovecot). These were solved only when I recreated the mail account on the client side so th eclient queried the server anew and created its settings anew. It’s not really debugging but trial and error, but then, debugging anything on iOS (anything Apple, really) is hard to do. G
spamsieve question
My new dovecot covers spam/ham training through the standard Spam_train and Notspam_train mailboxes. When a message is copied (I’m using macOS/iOS standard Apple clients) it triggers the spam/ham learning by rspamd. So far so good. When I move a message to Spam_train I’d like it to be removed immediately after learning. When I move a message to Notspam_train, I’d like it to move back where it came from after having been piped The first is I think doable, but how? Is the second doable? Gerben Wierda Chess and the Art of Enterprise Architecture <https://ea.rna.nl/the-book/> Mastering ArchiMate <https://ea.rna.nl/the-book-edition-iii/> Architecture for Real Enterprises <https://www.infoworld.com/blog/architecture-for-real-enterprises/> at InfoWorld On Slippery Ice <https://eapj.org/on-slippery-ice/> at EAPJ
Works, but why? Was: Trouble using reverse dsync backup (log in successful but almost nothing happens)
Answering my own question: removing the existing mailbox and running again worked. I don’t know why, I think it should have made the existing mailbox equivalent to the backupped one, shouldn’t it? Gerben Wierda Chess and the Art of Enterprise Architecture <http://enterprisechess.com/> Mastering ArchiMate <http://masteringarchimate.com/> Architecture for Real Enterprises <https://www.infoworld.com/blog/architecture-for-real-enterprises/> at InfoWorld On Slippery Ice <https://eapj.org/on-slippery-ice/> at EAPJ > On 23 Dec 2019, at 01:47, Gerben Wierda wrote: > > I am migrating an old dovecot 2.2.30 (macOS High SIerra, macO S Server) to a > new server with dovecot 2.3.7 (macOS Mojave, macports). > > To get the data over, I’m trying to do a reverse backup with doveadm between > an old and a new mail server, but failing to do so. I’m trying (from the new > server, the old server is 192.168.2.67): > > doveadm -Dv \ > -o imapc_host=192.168.2.67 \ > -o imapc_user=gerben \ > -o imapc_password=’[snip]' \ > -o imapc_features='rfc822.size fetch-headers’ \ > -o mail_prefetch_count=20 \ > -o mail_fsync=never \ > -o imapc_ssl=starttls \ > -o imapc_ssl_verify=no \ >backup -R -u gerben imapc: > > It seems to start to work: > > dsync(gerben): Debug: imapc(192.168.2.67:143): Created new connection > dsync(gerben): Debug: imapc(192.168.2.67:143): Looking up IP address > (reconnect_ok=true, last_connect=1577059834) > dsync(gerben): Debug: imapc(192.168.2.67:143): Connecting to 192.168.2.67:143 > dsync(gerben): Info: imapc(192.168.2.67:143): Connected to 192.168.2.67:143 > (local 192.168.2.66:50804) > dsync(gerben): Debug: imapc(192.168.2.67:143): Server capabilities: IMAP4rev1 > LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS LOGINDISABLED > XAPPLEPUSHSERVICE > dsync(gerben): Debug: imapc(192.168.2.67:143): Starting SSL handshake > dsync(gerben): Debug: imapc(192.168.2.67:143): SSL handshake successful, > ignoring invalid certificate: SSL certificate doesn't match expected host > name 192.168.2.67: No match to 2 SubjectAltNames > dsync(gerben): Debug: imapc(192.168.2.67:143): Server capabilities: IMAP4rev1 > LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE XAPPLEPUSHSERVICE > AUTH=CRAM-MD5 AUTH=PLAIN AUTH=LOGIN AUTH=DIGEST-MD5 AUTH=GSSAPI > dsync(gerben): Debug: imapc(192.168.2.67:143): Authenticating as gerben > dsync(gerben): Debug: imapc(192.168.2.67:143): Server capabilities: IMAP4rev1 > LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY > THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL > CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 > CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS > BINARY MOVE XAPPLEPUSHSERVICE COMPRESS=DEFLATE QUOTA ACL RIGHTS=texk > dsync(gerben): Debug: imapc(192.168.2.67:143): Authenticated successfully > dsync(gerben): Debug: imapc: root=, index=, indexpvt=, control=, inbox=, alt= > dsync(gerben): Debug: fts: Indexes disabled for namespace '' > dsync(gerben): Debug: brain S: Namespace has location imapc: > dsync(gerben): Debug: brain M: Local mailbox tree: INBOX > guid=184a4c066b219a5dd7379e3b11ac uid_validity=1570382187 uid_next=18 > subs=no last_change=0 last_subs=0 > dsync(gerben): Debug: brain S: Local mailbox tree: INBOX > guid=c92f64f79f0d1ed01e6d5b314f04886c uid_validity=1232523592 uid_next=482711 > subs=no last_change=0 last_subs=0 > dsync(gerben): Debug: brain M: Local mailbox tree: Notspam_train > guid=e0cc8c076b219a5dd7379e3b11ac uid_validity=1570382188 uid_next=10 > subs=no last_change=0 last_subs=0 > dsync(gerben): Debug: brain S: Local mailbox tree: zzz-Old > guid=c71fb29dcfd52e98ffb476033482311a uid_validity=1246525241 uid_next=1 > subs=yes last_change=0 last_subs=0 > dsync(gerben): Debug: brain M: Local mailbox tree: Sent > guid=e8c3b0076b219a5dd7379e3b11ac uid_validity=1570382189 uid_next=17 > subs=yes last_change=0 last_subs=1570382187 > dsync(gerben): Debug: brain S: Local mailbox tree: zzz-Old/Werk > guid=493c1b0ff8829799ae854dcdd35bdc49 uid_validity=1246525654 uid_next=1 > subs=yes last_change=0 last_subs=0 > dsync(gerben): Debug: brain M: Local mailbox tree: Spam_train > guid=7076d5076b219a5dd7379e3b11ac uid_validity=1570382190 uid_next=3719 > subs=no last_change=0 last_subs=0 > dsync(gerben): Debug: brain S: Local mailbox tree: > zzz-Old/Werk/UitNFIMailboxen guid=1da8c05ea358b80ee1c2f3a01cc7f2cc > uid_validity=1232523595 uid_next=118 subs=yes last_change=0 last_subs=0 > dsync(gerben): Debug: brain M: Local mailbox tree: Trash > guid=4810f5076b219a5dd7379e3b11ac uid_validity=1570382191 uid_next=3 > subs=no l
Trouble using reverse dsync backup (log in successful but almost nothing happens)
ug: brain S: Remote mailbox tree: Drafts guid=60cd72148903005e6b5501009e3b11ac uid_validity=1570382198 uid_next=1 subs=yes last_change=0 last_subs=1570382187 dsync(gerben): Debug: imapc(192.168.2.67:143): Disconnected But nothing has been copied over and while it states it has deleted local mailboxes (such as Subfolder/SUbSubFolder), they are still there. In other words: nothing happens. Well, not quite. The first time I ran this, local directory /var/mail/nl.rna.mail/rules/gerben/gerben/tmp was created. Can anyone see here what is going wrong? Gerben Wierda Chess and the Art of Enterprise Architecture <http://enterprisechess.com/> Mastering ArchiMate <http://masteringarchimate.com/> Architecture for Real Enterprises <https://www.infoworld.com/blog/architecture-for-real-enterprises/> at InfoWorld On Slippery Ice <https://eapj.org/on-slippery-ice/> at EAPJ
Re: Is it safe to mix dovecot import/backup/sync between dovecot versions 2.2.30.2 and 2.3.7.1?
Thank you. > On 22 Dec 2019, at 12:36, Sami Ketola wrote: > > >> On 22 Dec 2019, at 12.22, Gerben Wierda wrote: >> >> I am migrating a server. The old one uses dovecot 2.2.30.2 and the new one >> 2.3.7.1. I don’t want to use any tool that might influence flags on >> messages. I am using Maildir format on both ends and I have a way to let one >> server access the data on the other (either using an rsync or a direct >> access). >> >> Could I set up a sync between both to create a smooth migration? > > rsync probably works out fine, but I would use native dovecot dsync myself. So just for certainty: I canjust perfectly copy over the 2.2.30 Maildir to the new server for each user and dovecot 2.3.7 would just work? No need to do anything else, such as recreating indexes etc? That would probably be perfect for me. > > Something like this run on the new server: > > /usr/bin/doveadm backup -u -R ssh adminuser@old-server "sudo > /usr/bin/doveadm dsync-server -u ” For this I would have to create the ssh-setup for adminuser which I’re rather prevent to have to do, because the _dovecot user doesn’t have a login shell etc. and the servers are rather closed down with respect to ssh etc for security reasons. I have to admit I haven't really had to administer dovecot before (macOS Server was doing that for me) so I’m quite overwhelmed with all the settings, options, configurations and workings. It would be nice to have something like this in a crontab of the *old* server: doveadm sync remote:server-replica.example.com but I have no idea what I have to do in terms of getting this to work safely. Probably set up a master user on the new server’s end? Since all instructions I can find are based on some sort of Linux (and my servers are macOS) I have no idea yet what to do *exactly*. There are also no imapc entries yet on both servers in the config. User database on macOS is different as far as I know (LDAP: Open DIrectory). I am running into not understanding this, for instance from the Wiki: # Authenticate as masteruser / masteruser-secret, but use a separate login user. # If you don't have a master user, remove the imapc_master_user setting. imapc_user = %u imapc_master_user = masteruser imapc_password = masteruser-secret And that leaves me wondering: what does "imapc_user = %u" do exactly? Can I use any existing local user to run the doveadm commands and will the %u become the user name of that user for instance? I other words, I’m drowning in details and a simple copy of the mail directories would be a perfect solution. G
Is it safe to mix dovecot import/backup/sync between dovecot versions 2.2.30.2 and 2.3.7.1?
I am migrating a server. The old one uses dovecot 2.2.30.2 and the new one 2.3.7.1. I don’t want to use any tool that might influence flags on messages. I am using Maildir format on both ends and I have a way to let one server access the data on the other (either using an rsync or a direct access). Could I set up a sync between both to create a smooth migration? Gerben Wierda Chess and the Art of Enterprise Architecture <http://enterprisechess.com/> Mastering ArchiMate <http://masteringarchimate.com/> Architecture for Real Enterprises <https://www.infoworld.com/blog/architecture-for-real-enterprises/> at InfoWorld On Slippery Ice <https://eapj.org/on-slippery-ice/> at EAPJ
Is it possible to create and use some lookup table to replace (for instance) %u in mail_location?
Suppose my mail_location is: mail_location = maildir:/private/var/mail/nl.rna.mail/%u But I would like to use a simple lookup table to replace %u with another value, say '8DB4E345-9144-4F92-8C9F-9D47BB61CBF8’ instead of ‘username’. Is there a (simple) way to do this? Gerben Wierda Chess and the Art of Enterprise Architecture <http://enterprisechess.com/> Mastering ArchiMate <http://masteringarchimate.com/> Architecture for Real Enterprises <https://www.infoworld.com/blog/architecture-for-real-enterprises/> at InfoWorld On Slippery Ice <https://eapj.org/on-slippery-ice/> at EAPJ
Re: New to dovecot admin, question about using LDAP for user-specific values
> On 13 Sep 2019, at 11:51, Jean-Daniel Dupas wrote: > > > >> Le 13 sept. 2019 à 09:29, Gerben Wierda via dovecot > <mailto:dovecot@dovecot.org>> a écrit : >> >> Nobody? >> >>> On 10 Sep 2019, at 11:58, Gerben Wierda via dovecot >> <mailto:dovecot@dovecot.org>> wrote: >>> >>> I am new to dovecot administration. I’ve read the Wiki but that hasn’t >>> given me the understanding I need. >>> >>> When I query my LDAP (on macOS) on a value for user ‘gerben’, I can get >>> that: >>> >>> dumbledore:~ gerben$ dscl /LDAPv3/127.0.0.1 -read /users/gerben GeneratedUID >>> GeneratedUID: 780D870E-6B00-478E-AB70-3D3307215A82 >>> >>> I would like to use that value in dovecot settings, e.g. something like >>> >>> user_attrs = \ >>> =mail=maildir://Library/Server/Mail/Data/mail/%{ldap:GeneratedUID} >>> >>> >>> Is this possible and if so what do I exactly need to do to get this working? > > As the answer is in the question, it is hard to give you any hint about what > should be done. > > What is wrong with > > user_attrs = \ > =mail=maildir://Library/Server/Mail/Data/mail/%{ldap:GeneratedUID} > > > Did you try it ? Have you got any issue with it ? I haven’t tried anything yet as I am trying to learn before I do anything (and trial and error is a very slow method), but it seems to me that just that line cannot be enough. Because how does “ldap:” know to go looking in the LDAP structure at "/Users//“ in the LDAP “/LDAPv3/127.0.0.1”? Somehow I shall have to tell dovecot that. But if trial and error is the only way, I’ll have to try. G
Re: New to dovecot admin, question about using LDAP for user-specific values
Nobody? > On 10 Sep 2019, at 11:58, Gerben Wierda via dovecot > wrote: > > I am new to dovecot administration. I’ve read the Wiki but that hasn’t given > me the understanding I need. > > When I query my LDAP (on macOS) on a value for user ‘gerben’, I can get that: > > dumbledore:~ gerben$ dscl /LDAPv3/127.0.0.1 -read /users/gerben GeneratedUID > GeneratedUID: 780D870E-6B00-478E-AB70-3D3307215A82 > > I would like to use that value in dovecot settings, e.g. something like > > user_attrs = \ > =mail=maildir://Library/Server/Mail/Data/mail/%{ldap:GeneratedUID} > > > Is this possible and if so what do I exactly need to do to get this working? > > Gerben Wierda > Chess and the Art of Enterprise Architecture <https://ea.rna.nl/the-book/> > Mastering ArchiMate <https://ea.rna.nl/the-book-edition-iii/> > Architecture for Real Enterprises > <https://www.infoworld.com/blog/architecture-for-real-enterprises/> at > InfoWorld > On Slippery Ice <https://eapj.org/on-slippery-ice/> at EAPJ >
New to dovecot admin, question about testing a dovecot configuration
I am new to dovecot administration, not new to postfix administration. When setting up a new postfix/dovecot combo from scratch, can I start with dovecot alone and still be able to test it? E.g. can I address the local delivery agent with some sort of test email? If so, how? I haven’t been able to find it in the wiki. Gerben Wierda Chess and the Art of Enterprise Architecture <https://ea.rna.nl/the-book/> Mastering ArchiMate <https://ea.rna.nl/the-book-edition-iii/> Architecture for Real Enterprises <https://www.infoworld.com/blog/architecture-for-real-enterprises/> at InfoWorld On Slippery Ice <https://eapj.org/on-slippery-ice/> at EAPJ
New to dovecot admin, question about using LDAP for user-specific values
I am new to dovecot administration. I’ve read the Wiki but that hasn’t given me the understanding I need. When I query my LDAP (on macOS) on a value for user ‘gerben’, I can get that: dumbledore:~ gerben$ dscl /LDAPv3/127.0.0.1 -read /users/gerben GeneratedUID GeneratedUID: 780D870E-6B00-478E-AB70-3D3307215A82 I would like to use that value in dovecot settings, e.g. something like user_attrs = \ =mail=maildir://Library/Server/Mail/Data/mail/%{ldap:GeneratedUID} Is this possible and if so what do I exactly need to do to get this working? Gerben Wierda Chess and the Art of Enterprise Architecture <https://ea.rna.nl/the-book/> Mastering ArchiMate <https://ea.rna.nl/the-book-edition-iii/> Architecture for Real Enterprises <https://www.infoworld.com/blog/architecture-for-real-enterprises/> at InfoWorld On Slippery Ice <https://eapj.org/on-slippery-ice/> at EAPJ
Checking sieve
A simple question: I’ve installed roundcube on macOS Sierra with Server 5.2 and created a set of sieve rules in my dovecot setup. How can I check their functioning (other than send test mails) e.g. is there some logging I can set / look at? G
Managing Sieve on macOS 10.12 Sierra with Server 5.2?
I don’t know if this is considered OT, but dovecot has support for sieve. I had a rule set running under Mac OS X 10.8.5 Server 2.2.5 which I did maintain by hand on the server because a while before Mac OS X Server had dropped support for Squirrelmail. Now I wonder if there is a decent way to get sieve running again on macOS 10.12 Sierra with Server 5.2 G
Maildir: do message files need to have a unique name across all directories?
While testing a new mail server I created a temporary account and filled its mailbox (Milder, dovecot) with the contents of another account. That means the actual message files in each account have identical names. Is that a problem? (I’m asking because I see some weird behaviour of visible/invisible messages and I wonder if this could have been the reason). G
Re: Messed up dovecot mail store, need some repair advice
> On 22 Oct 2016, at 22:09, Gerben Wierda wrote: > >> >> On 22 Oct 2016, at 19:04, Aki Tuomi wrote: >> >> >>> On October 22, 2016 at 7:51 PM Gerben Wierda wrote: >>> >>> >>> Hello folks, >>> >>> I have an older dovecot 2.0 (which I will migrate to a 2.2 asap, but at >>> this point in time I need a fix). This is a dovecot 2.0 that came with Mac >>> OS X 10.8.5 Server 2.2.5. >>> >>> Today, my spam/virus filtering (clamav) on the Server broke down. As a >>> result, all my meesages got the ***UNCHECKED*** tag added to each subject. >>> That was clearly unacceptable. SO, for the tim ebeing I have set that tag >>> to undef so the tag is no longer added. But I also wanted to repair the >>> messages that already ended up in dovecot 2.0 >>> >>> So, I did something simple: stopped all mail services on the server, went >>> into the dovecot mail store and edited the messages. I first tried with one >>> small ccount and it seemed OK. But now my mail client is experiencing >>> problems with the messages (cannot display) and I think I've been to >>> simplistic. I have for instance notedthat th esizeof the message is part of >>> the filename. So, I can change these of course, but probably I need to >>> change more. >>> >>> Can someone enlighten me how I can repair the broken data store? >>> >>> Thanks, >>> >>> (Foolish) Gerben >> >> I think your best bet is to delete dovecot.index* and run dovecot index -u >> username. Or revert all your changes. Depending what mail store you are >> using, this might lose any flags on your mails, such as \Read. > > Thanks. Losing 300 flags and unread on thousands of emails was not a > preferred scenario. > > I was able to repair by > - turning dovecote (and other mail services) off > - find all message files that were changed in a certain period > - check all their names against their file sizes (this found me the edited > ones) > - returning the ***UNCHECKED*** string to the Subject lines making the file > sizes equal to the size as reported in the name of the file > > Which leaves me with something I would really like: change the subject line > of 5-10 messages in dovecot, without destroying everything. > > I was thinking about the following scenario: > - create a separate mailbox REPAIR within user X’s mail store (the INBOX, > btw, is named ‘cur’) > - move all to be changed messages there using the mail client > - kill the mail client > - stop dovecot > - edit the messages and change the names of the files so the S= W= parts are > in line with the new content. (I understand S, but what is W?) > - run “devoid index -u user REPAIR” > - start dovecot > - start email client (potentially, reload the entire mail store for that user) > > Would that work? There was an easier solution. In my mail program I created a local mailbox, copied the messages there, edited them on disk, rebuilt the local mailbox and then moved them back to IMAP. G
Re: Messed up dovecot mail store, need some repair advice
> On 22 Oct 2016, at 19:04, Aki Tuomi wrote: > > >> On October 22, 2016 at 7:51 PM Gerben Wierda wrote: >> >> >> Hello folks, >> >> I have an older dovecot 2.0 (which I will migrate to a 2.2 asap, but at this >> point in time I need a fix). This is a dovecot 2.0 that came with Mac OS X >> 10.8.5 Server 2.2.5. >> >> Today, my spam/virus filtering (clamav) on the Server broke down. As a >> result, all my meesages got the ***UNCHECKED*** tag added to each subject. >> That was clearly unacceptable. SO, for the tim ebeing I have set that tag to >> undef so the tag is no longer added. But I also wanted to repair the >> messages that already ended up in dovecot 2.0 >> >> So, I did something simple: stopped all mail services on the server, went >> into the dovecot mail store and edited the messages. I first tried with one >> small ccount and it seemed OK. But now my mail client is experiencing >> problems with the messages (cannot display) and I think I've been to >> simplistic. I have for instance notedthat th esizeof the message is part of >> the filename. So, I can change these of course, but probably I need to >> change more. >> >> Can someone enlighten me how I can repair the broken data store? >> >> Thanks, >> >> (Foolish) Gerben > > I think your best bet is to delete dovecot.index* and run dovecot index -u > username. Or revert all your changes. Depending what mail store you are > using, this might lose any flags on your mails, such as \Read. Thanks. Losing 300 flags and unread on thousands of emails was not a preferred scenario. I was able to repair by - turning dovecote (and other mail services) off - find all message files that were changed in a certain period - check all their names against their file sizes (this found me the edited ones) - returning the ***UNCHECKED*** string to the Subject lines making the file sizes equal to the size as reported in the name of the file Which leaves me with something I would really like: change the subject line of 5-10 messages in dovecot, without destroying everything. I was thinking about the following scenario: - create a separate mailbox REPAIR within user X’s mail store (the INBOX, btw, is named ‘cur’) - move all to be changed messages there using the mail client - kill the mail client - stop dovecot - edit the messages and change the names of the files so the S= W= parts are in line with the new content. (I understand S, but what is W?) - run “devoid index -u user REPAIR” - start dovecot - start email client (potentially, reload the entire mail store for that user) Would that work? G
Messed up dovecot mail store, need some repair advice
Hello folks, I have an older dovecot 2.0 (which I will migrate to a 2.2 asap, but at this point in time I need a fix). This is a dovecot 2.0 that came with Mac OS X 10.8.5 Server 2.2.5. Today, my spam/virus filtering (clamav) on the Server broke down. As a result, all my meesages got the ***UNCHECKED*** tag added to each subject. That was clearly unacceptable. SO, for the tim ebeing I have set that tag to undef so the tag is no longer added. But I also wanted to repair the messages that already ended up in dovecot 2.0 So, I did something simple: stopped all mail services on the server, went into the dovecot mail store and edited the messages. I first tried with one small ccount and it seemed OK. But now my mail client is experiencing problems with the messages (cannot display) and I think I've been to simplistic. I have for instance notedthat th esizeof the message is part of the filename. So, I can change these of course, but probably I need to change more. Can someone enlighten me how I can repair the broken data store? Thanks, (Foolish) Gerben
Migrating users from a 2.0.19 to a 2.2.24 installation
Hello, I am currently still running an older dovecot (2.0.19apple1 on Mac OS X 10.8.5) and I want to migrate my users to a new server (macOS 10.12 with Server 5, which contains dovecot 2.2.24 (a82c823)). Basically, I want to create a new server installation on the new server so I don't bring any junk over (new user accounts, with the same uid/gid (still need to figure that one out), but after I have done that I need to move the data over from the old instalation to the new. Has anything changed in the formats between 2.0 and 2.2 that will stop me from doing this? Thanks, G