Re: Dovecot does not start on MacOS 12.01

[Steven Smith]

I plan to issue a PR to MacPorts’s dovecot port with the patch below on macOS 
12.

Is anyone able to offer a better workaround, or explain the pitfalls of 
removing these non-working lines?

Patch:

> --- ./src/lib/restrict-process-size.c 2021-12-19 11:46:50.0 -0500
> +++ ./src/lib/restrict-process-size.c 2021-12-19 11:49:13.0 -0500
> @@ -9,18 +9,24 @@
>  {
>   struct rlimit rlim;
>  
> - rlim.rlim_max = rlim.rlim_cur = bytes;
> - if (setrlimit(RLIMIT_DATA, &rlim) < 0) {
> - i_fatal("setrlimit(RLIMIT_DATA, %llu): %m",
> - (unsigned long long)bytes);
> - }
>  
> -#ifdef HAVE_RLIMIT_AS
> - if (setrlimit(RLIMIT_AS, &rlim) < 0) {
> - i_fatal("setrlimit(RLIMIT_AS, %llu): %m",
> - (unsigned long long)bytes);
> - }
> -#endif
> +/*
> + * Workaround for issue with dovercot version 2.3.17 on macOS 12
> + * https://www.mail-archive.com/dovecot@dovecot.org/msg84784.html
> + * 
> + *   rlim.rlim_max = rlim.rlim_cur = bytes;
> + *   if (setrlimit(RLIMIT_DATA, &rlim) < 0) {
> + *   i_fatal("setrlimit(RLIMIT_DATA, %llu): %m",
> + *   (unsigned long long)bytes);
> + *   }
> + * 
> + * #ifdef HAVE_RLIMIT_AS
> + *   if (setrlimit(RLIMIT_AS, &rlim) < 0) {
> + *   i_fatal("setrlimit(RLIMIT_AS, %llu): %m",
> + *   (unsigned long long)bytes);
> + *   }
> + * #endif
> +*/
>  }
>  
>  void restrict_process_count(rlim_t count ATTR_UNUSED)




> On Dec 19, 2021, at 12:00 PM, Steven Smith  wrote:
> 
> I confirm that a working workaround on macOS 12 is to simply remove these 
> lines from ./src/lib/restrict-process-size.c:
> 
> https://github.com/dovecot/core/blob/a5209c83c3a82386c94d466eec5fea394973e88f/src/lib/restrict-process-size.c#L12-L23
>  
> <https://github.com/dovecot/core/blob/a5209c83c3a82386c94d466eec5fea394973e88f/src/lib/restrict-process-size.c#L12-L23>
> 
> 
>> On Dec 19, 2021, at 11:26 AM, Steven Smith > <mailto:steve.t.sm...@gmail.com>> wrote:
>> 
>> Re: https://www.mail-archive.com/dovecot@dovecot.org/msg84784.html 
>> <https://www.mail-archive.com/dovecot@dovecot.org/msg84784.html>
>> 
>> I just upgraded to macOS 12.1, ran into this issue, and discovered this 
>> thread from the error message.
>> 
>> Is there a viable workaround or patch yet?
>> 
>> I tried the workaround in this thread, and still see these issues.
>> 
>> Set default_vsz_limit = 0 in ./conf.d/10-master.conf.
>> 
>> Observe that:
>> dovecot now launches without the setrlimit(RLIMIT_DATA, 268435456) error at 
>> the command line
>> However, the mail-err.log is still filled with these errors (below), perhaps 
>> for the child processes.
>> 
>> Steve
>> 
>> 
>>> Dec 19 11:22:06 imap-login: Error: Dec 19 11:22:06 service(imap-login): 
>>> Fatal: setrlimit(RLIMIT_DATA, 2147483648): Invalid argument
>>> Dec 19 11:22:06 imap-login: Error: Dec 19 11:22:06 service(imap-login): 
>>> Fatal: setrlimit(RLIMIT_DATA, 2147483648): Invalid argument
>>> Dec 19 11:22:06 imap-login: Error: Dec 19 11:22:06 service(imap-login): 
>>> Fatal: setrlimit(RLIMIT_DATA, 2147483648): Invalid argument
>>> Dec 19 11:22:06 imap-login: Error: Dec 19 11:22:06 service(imap-login): 
>>> Fatal: setrlimit(RLIMIT_DATA, 2147483648): Invalid argument
>>> Dec 19 11:22:06 imap-login: Error: Dec 19 11:22:06 service(imap-login): 
>>> Fatal: setrlimit(RLIMIT_DATA, 2147483648): Invalid argument
>>> Dec 19 11:22:06 imap-login: Fatal: master: service(imap-login): child 4629 
>>> returned error 89 (Fatal failure)
>>> Dec 19 11:22:06 master: Error: service(imap-login): command startup failed, 
>>> throttling for 2.000 secs
>>> Dec 19 11:22:06 imap-login: Fatal: master: service(imap-login): child 4630 
>>> returned error 89 (Fatal failure)
>>> Dec 19 11:22:06 imap-login: Fatal: master: service(imap-login): child 4628 
>>> returned error 89 (Fatal failure)
>>> Dec 19 11:22:06 imap-login: Fatal: master: service(imap-login): child 4627 
>>> returned error 89 (Fatal failure)
>>> Dec 19 11:22:06 imap-login: Fatal: master: service(imap-login): child 4619 
>>> returned error 89 (Fatal failure)
>>> Dec 19 11:22:06 imap-login: Error: Dec 19 11:22:06 service(imap-login): 
>>> Fatal: setrlimit(RLIMIT_DATA, 2147483648): Invalid argument
>>> Dec 19 11:22:06 imap-login: Fatal: master: service(imap-login): child 4631 
>>> returned error 89 (Fatal failure)
>>> Dec 19 11:22:09 imap-login: Error: Dec 19 11:22:09 service(imap-login): 
>>> Fatal: setrlimit(RLIMIT_DATA, 2147483648): Invalid argument
>>> Dec 19 11:22:09 imap-login: Fatal: master: service(imap-login): child 4638 
>>> returned error 89 (Fatal failure)
>>> Dec 19 11:22:09 master: Error: service(imap-login): command startup failed, 
>>> throttling for 4.000 secs
>> 
>> 
>> 
>> 
>>> On Mon, 1 Nov 2021 14:55:25 +0200, Aki Tuomi said:
>>> 
>>> I think this is happening because MacOS no longer supports RLIMIT_DATA. 
>>> We'll 
>>> try to look into this. default_vsz_limit = 0 seems good enough workaround.
>>> Aki
>> 
>> 
> 

Re: Dovecot does not start on MacOS 12.01

[Steven Smith]

I confirm that a working workaround on macOS 12 is to simply remove these lines 
from ./src/lib/restrict-process-size.c:

https://github.com/dovecot/core/blob/a5209c83c3a82386c94d466eec5fea394973e88f/src/lib/restrict-process-size.c#L12-L23


> On Dec 19, 2021, at 11:26 AM, Steven Smith  wrote:
> 
> Re: https://www.mail-archive.com/dovecot@dovecot.org/msg84784.html 
> <https://www.mail-archive.com/dovecot@dovecot.org/msg84784.html>
> 
> I just upgraded to macOS 12.1, ran into this issue, and discovered this 
> thread from the error message.
> 
> Is there a viable workaround or patch yet?
> 
> I tried the workaround in this thread, and still see these issues.
> 
> Set default_vsz_limit = 0 in ./conf.d/10-master.conf.
> 
> Observe that:
> dovecot now launches without the setrlimit(RLIMIT_DATA, 268435456) error at 
> the command line
> However, the mail-err.log is still filled with these errors (below), perhaps 
> for the child processes.
> 
> Steve
> 
> 
>> Dec 19 11:22:06 imap-login: Error: Dec 19 11:22:06 service(imap-login): 
>> Fatal: setrlimit(RLIMIT_DATA, 2147483648): Invalid argument
>> Dec 19 11:22:06 imap-login: Error: Dec 19 11:22:06 service(imap-login): 
>> Fatal: setrlimit(RLIMIT_DATA, 2147483648): Invalid argument
>> Dec 19 11:22:06 imap-login: Error: Dec 19 11:22:06 service(imap-login): 
>> Fatal: setrlimit(RLIMIT_DATA, 2147483648): Invalid argument
>> Dec 19 11:22:06 imap-login: Error: Dec 19 11:22:06 service(imap-login): 
>> Fatal: setrlimit(RLIMIT_DATA, 2147483648): Invalid argument
>> Dec 19 11:22:06 imap-login: Error: Dec 19 11:22:06 service(imap-login): 
>> Fatal: setrlimit(RLIMIT_DATA, 2147483648): Invalid argument
>> Dec 19 11:22:06 imap-login: Fatal: master: service(imap-login): child 4629 
>> returned error 89 (Fatal failure)
>> Dec 19 11:22:06 master: Error: service(imap-login): command startup failed, 
>> throttling for 2.000 secs
>> Dec 19 11:22:06 imap-login: Fatal: master: service(imap-login): child 4630 
>> returned error 89 (Fatal failure)
>> Dec 19 11:22:06 imap-login: Fatal: master: service(imap-login): child 4628 
>> returned error 89 (Fatal failure)
>> Dec 19 11:22:06 imap-login: Fatal: master: service(imap-login): child 4627 
>> returned error 89 (Fatal failure)
>> Dec 19 11:22:06 imap-login: Fatal: master: service(imap-login): child 4619 
>> returned error 89 (Fatal failure)
>> Dec 19 11:22:06 imap-login: Error: Dec 19 11:22:06 service(imap-login): 
>> Fatal: setrlimit(RLIMIT_DATA, 2147483648): Invalid argument
>> Dec 19 11:22:06 imap-login: Fatal: master: service(imap-login): child 4631 
>> returned error 89 (Fatal failure)
>> Dec 19 11:22:09 imap-login: Error: Dec 19 11:22:09 service(imap-login): 
>> Fatal: setrlimit(RLIMIT_DATA, 2147483648): Invalid argument
>> Dec 19 11:22:09 imap-login: Fatal: master: service(imap-login): child 4638 
>> returned error 89 (Fatal failure)
>> Dec 19 11:22:09 master: Error: service(imap-login): command startup failed, 
>> throttling for 4.000 secs
> 
> 
> 
> 
>> On Mon, 1 Nov 2021 14:55:25 +0200, Aki Tuomi said:
>> 
>> I think this is happening because MacOS no longer supports RLIMIT_DATA. 
>> We'll 
>> try to look into this. default_vsz_limit = 0 seems good enough workaround.
>> Aki
> 
> 

Re: Dovecot does not start on MacOS 12.01

[Steven Smith]

Re: https://www.mail-archive.com/dovecot@dovecot.org/msg84784.html 


I just upgraded to macOS 12.1, ran into this issue, and discovered this thread 
from the error message.

Is there a viable workaround or patch yet?

I tried the workaround in this thread, and still see these issues.

Set default_vsz_limit = 0 in ./conf.d/10-master.conf.

Observe that:
dovecot now launches without the setrlimit(RLIMIT_DATA, 268435456) error at the 
command line
However, the mail-err.log is still filled with these errors (below), perhaps 
for the child processes.

Steve


> Dec 19 11:22:06 imap-login: Error: Dec 19 11:22:06 service(imap-login): 
> Fatal: setrlimit(RLIMIT_DATA, 2147483648): Invalid argument
> Dec 19 11:22:06 imap-login: Error: Dec 19 11:22:06 service(imap-login): 
> Fatal: setrlimit(RLIMIT_DATA, 2147483648): Invalid argument
> Dec 19 11:22:06 imap-login: Error: Dec 19 11:22:06 service(imap-login): 
> Fatal: setrlimit(RLIMIT_DATA, 2147483648): Invalid argument
> Dec 19 11:22:06 imap-login: Error: Dec 19 11:22:06 service(imap-login): 
> Fatal: setrlimit(RLIMIT_DATA, 2147483648): Invalid argument
> Dec 19 11:22:06 imap-login: Error: Dec 19 11:22:06 service(imap-login): 
> Fatal: setrlimit(RLIMIT_DATA, 2147483648): Invalid argument
> Dec 19 11:22:06 imap-login: Fatal: master: service(imap-login): child 4629 
> returned error 89 (Fatal failure)
> Dec 19 11:22:06 master: Error: service(imap-login): command startup failed, 
> throttling for 2.000 secs
> Dec 19 11:22:06 imap-login: Fatal: master: service(imap-login): child 4630 
> returned error 89 (Fatal failure)
> Dec 19 11:22:06 imap-login: Fatal: master: service(imap-login): child 4628 
> returned error 89 (Fatal failure)
> Dec 19 11:22:06 imap-login: Fatal: master: service(imap-login): child 4627 
> returned error 89 (Fatal failure)
> Dec 19 11:22:06 imap-login: Fatal: master: service(imap-login): child 4619 
> returned error 89 (Fatal failure)
> Dec 19 11:22:06 imap-login: Error: Dec 19 11:22:06 service(imap-login): 
> Fatal: setrlimit(RLIMIT_DATA, 2147483648): Invalid argument
> Dec 19 11:22:06 imap-login: Fatal: master: service(imap-login): child 4631 
> returned error 89 (Fatal failure)
> Dec 19 11:22:09 imap-login: Error: Dec 19 11:22:09 service(imap-login): 
> Fatal: setrlimit(RLIMIT_DATA, 2147483648): Invalid argument
> Dec 19 11:22:09 imap-login: Fatal: master: service(imap-login): child 4638 
> returned error 89 (Fatal failure)
> Dec 19 11:22:09 master: Error: service(imap-login): command startup failed, 
> throttling for 4.000 secs




> On Mon, 1 Nov 2021 14:55:25 +0200, Aki Tuomi said:
> 
> I think this is happening because MacOS no longer supports RLIMIT_DATA. We'll 
> try to look into this. default_vsz_limit = 0 seems good enough workaround.
> Aki




smime.p7s
Description: S/MIME cryptographic signature

Re: Dovecot Subfolder Always Uses dot separator with mdbox

[Steven Smith]

This is resolved by deleting the Mail account on macOS, then adding it back.

Apparently macOS Mail.app must cache these settings at account creation time.

The correct settings are:

> mail_location = mdbox:/private/var/mail/tld.hostname.mail/%Ln/mdbox:LAYOUT=fs
> namespace inbox {
> …
> separator = /
> …
> }

Re: Dovecot Subfolder Always Uses dot separator with mdbox

[Steven Smith]

Thank you very much!

Here’s what I see:


# separator =   [this line commented out in 10-mail.conf]

> 1 SELECT INBOX
> imap(pid 35573 user target-user): Debug: Mailbox INBOX: Mailbox opened 
> because: SELECT
> * FLAGS (\Answered \Flagged \Deleted \Seen \Draft $NotJunk NotJunk $Forwarded 
> Redirected $MailFlagBit0 Forwarded $MailFlagBit2 $MailFlagBit1)
> * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft $NotJunk 
> NotJunk $Forwarded Redirected $MailFlagBit0 Forwarded $MailFlagBit2 
> $MailFlagBit1 \*)] Flags permitted.
> * 21627 EXISTS
> * 0 RECENT
> * OK [UNSEEN 14430] First unseen.
> * OK [UIDVALIDITY 1558435823] UIDs valid
> * OK [UIDNEXT 22415] Predicted next UID
> * OK [HIGHESTMODSEQ 39411] Highest
> 1 OK [READ-WRITE] Select completed (0.001 + 0.000 secs).


separator = /

> imap(pid 35890 user target-user): Debug: Mailbox INBOX: Mailbox opened 
> because: SELECT
> * FLAGS (\Answered \Flagged \Deleted \Seen \Draft $NotJunk NotJunk $Forwarded 
> Redirected $MailFlagBit0 Forwarded $MailFlagBit2 $MailFlagBit1)
> * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft $NotJunk 
> NotJunk $Forwarded Redirected $MailFlagBit0 Forwarded $MailFlagBit2 
> $MailFlagBit1 \*)] Flags permitted.
> * 21627 EXISTS
> * 0 RECENT
> * OK [UNSEEN 14430] First unseen.
> * OK [UIDVALIDITY 1558435823] UIDs valid
> * OK [UIDNEXT 22415] Predicted next UID
> * OK [HIGHESTMODSEQ 39411] Highest
> 1 OK [READ-WRITE] Select completed (0.001 + 0.000 secs).



Steve


> On May 24, 2021, at 1:56 AM, Aki Tuomi  wrote:
> 
>> 
>> On 22/05/2021 20:36 Steven Smith  wrote:
>> 
>> 
>> I’m trying to configure dovecot to use mailbox subfolders that look like 
>> both foo/bar on the IMAP client and the filesystem.
>> 
>> But the subfolders created all look like foo.bar, not foo/bar with a proper 
>> nesting structure.
>> 
>> This also happens when the folder ‘bar’ is moved into the folder ‘foo’—A new 
>> folder `foo.bar’ is created.
>> 
>> I believe that i’m following the configuration rules at 
>> https://wiki.dovecot.org/MailboxFormat/Maildir#Maildir_and_filesystems and 
>> https://wiki.dovecot.org/MailLocation/Maildir. This issue happens whether I 
>> set separator to its default, ‘.’, or ‘/’:
>> 
> 
> Namespace separator and filesystem separator are different. It should be 
> using / as *filesystem* separator with LAYOUT=FS, and it should be using 
> namespace separator / since you've specified it.
> 
> Can you do `doveadm exec imap -o mail_debug=yes -u target-user` and provide 
> logs after `1 SELECT INBOX`
> 
> Aki



smime.p7s
Description: S/MIME cryptographic signature

Dovecot Subfolder Always Uses dot separator with mdbox

[Steven Smith]

I’m trying to configure dovecot to use mailbox subfolders that look like both 
foo/bar on the IMAP client and the filesystem.

But the subfolders created all look like foo.bar, not foo/bar with a proper 
nesting structure.

This also happens when the folder ‘bar’ is moved into the folder ‘foo’—A new 
folder `foo.bar’ is created.

I believe that i’m following the configuration rules at 
https://wiki.dovecot.org/MailboxFormat/Maildir#Maildir_and_filesystems and 
https://wiki.dovecot.org/MailLocation/Maildir. This issue happens whether I set 
separator to its default, ‘.’, or ‘/’:

> mail_location = mdbox:/private/var/mail/tld.hostname.mail/%Ln/mdbox:LAYOUT=fs
> namespace inbox {
>  …
>  separator = /
>  …
> }


The IMAP client is macOS Mail.app version 14.0.

I’d greatly appreciate any help or pointers to fix this configuration issue.


dovecot -n
> # 2.3.13 (): /opt/local/etc/dovecot/dovecot.conf
> # Pigeonhole version 0.5.13 ()
> # OS: Darwin 20.4.0 x86_64  apfs
> # Hostname: mail.hostname.tld
> aps_topic = com.apple.mail.XServer.--fff-ff
> auth_cache_size = 10 M
> auth_gssapi_hostname = $ALL
> auth_krb5_keytab = /opt/local/etc/dovecot/imap.keytab
> auth_mechanisms = plain gssapi
> auth_realms =mail.hostname.tld
> auth_socket_path = /opt/local/var/run/dovecot/auth-userdb
> auth_username_format = %Ln
> debug_log_path = /opt/local/var/log/mail/mail-debug.log
> disable_plaintext_auth = no
> first_valid_gid = 6
> first_valid_uid = 6
> imap_id_log = *
> imap_id_send = "name" * "version" *
> imap_idle_notify_interval = 29 mins
> imap_urlauth_submit_user = submit
> info_log_path = /opt/local/var/log/mail/mail-info.log
> last_valid_gid = 100
> lda_mailbox_autocreate = yes
> log_path = /opt/local/var/log/mail/mail-err.log
> login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c
> mail_access_groups = mail
> mail_attachment_dir = /private/var/mail/tld.hostname.mail/attachments
> mail_attachment_fs = sis posix:mode=0666
> mail_gid = mail
> mail_home = /private/var/mail/tld.hostname.mail
> mail_location = mdbox:/private/var/mail/tld.hostname.mail/%Ln/mdbox:LAYOUT=fs
> mail_log_prefix = "%s(pid %p user %u): "
> mail_plugins = quota zlib acl fts fts_solr fts_lucene
> mail_privileged_group = mail
> mail_uid = _dovecot
> managesieve_notify_capability = mailto
> managesieve_sieve_capability = fileinto reject envelope encoded-character 
> vacation subaddress comparator-i;ascii-numeric relational regex imap4flags 
> copy include variables body enotify environment mailbox date index ihave 
> duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve
> mdbox_rotate_size = 200 M
> namespace inbox {
>  inbox = yes
>  location = 
>  mailbox Archive {
>auto = subscribe
>special_use = \Archive
>  }
>  mailbox Drafts {
>auto = subscribe
>special_use = \Drafts
>  }
>  mailbox Junk {
>auto = create
>special_use = \Junk
>  }
>  mailbox Notspam_train {
>auto = create
>special_use = \Junk
>  }
>  mailbox Sent {
>auto = subscribe
>special_use = \Sent
>  }
>  mailbox "Sent Messages" {
>special_use = \Sent
>  }
>  mailbox Spam_train {
>auto = create
>special_use = \Junk
>  }
>  mailbox Trash {
>auto = create
>special_use = \Trash
>  }
>  prefix = 
>  separator = /
> }
> passdb {
>  driver = pam
>  name = pam
> }
> plugin {
>  fts = solr
>  fts_autoindex = yes
>  fts_autoindex_exclude = \Junk
>  fts_solr = url=http://127.0.0.1:8983/solr/dovecot/
>  imapsieve_mailbox1_before = 
> file:/opt/local/etc/dovecot/sieve/report-spam.sieve
>  imapsieve_mailbox1_causes = COPY APPEND
>  imapsieve_mailbox1_name = Spam_train
>  imapsieve_mailbox2_before = 
> file:/opt/local/etc/dovecot/sieve/report-ham.sieve
>  imapsieve_mailbox2_causes = COPY APPEND
>  imapsieve_mailbox2_name = Notspam_train
>  mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename
>  mail_log_fields = uid box msgid from subject size flags
>  quota_grace = 10%%
>  quota_rule = *:storage=16G
>  quota_rule2 = Trash:storage=+256M
>  quota_warning = storage=100%% quota-exceeded %u
>  quota_warning2 = storage=80%% quota-warning %u
>  recipient_delimiter = +
>  sieve = /private/var/mail/tld.hostname.mail/rules/%Ln/dovecot.sieve
>  sieve_after = /opt/local/etc/dovecot/sieve-after.d
>  sieve_before = /opt/local/etc/dovecot/sieve-before.d
>  sieve_dir = /private/var/mail/tld.hostname.mail/rules/%Ln/%u
>  sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment
>  sieve_pipe_bin_dir = /opt/local/etc/dovecot/sieve
>  sieve_plugins = sieve_imapsieve sieve_extprograms
>  sieve_quota_max_storage = 50M
> }
> postmaster_address = postmas...@hostname.tld
> protocols = imap lmtp sieve
> quota_full_tempfail = yes
> sendmail_path = /opt/local/sbin/sendmail
> service auth-worker {
>  group = mail
>  user = root
> }
> service auth {
>  extra_groups = _keytabusers
>  idle_kill = 15 mins
>  unix_listener /opt/local/var/spool/postfix/private/auth {
>group = mail
>  

Dovecot Subfolder Always Uses dot separator with mdbox

[Steven Smith]

I’m trying to configure dovecot to use mailbox subfolders that look like both 
foo/bar on the IMAP client and the filesystem.

But the subfolders created all look like foo.bar, not foo/bar with a proper 
nesting structure.

This also happens when the folder ‘bar’ is moved into the folder ‘foo’—A new 
folder `foo.bar’ is created.

I believe that i’m following the configuration rules at 
https://wiki.dovecot.org/MailboxFormat/Maildir#Maildir_and_filesystems and 
https://wiki.dovecot.org/MailLocation/Maildir. This issue happens whether I set 
separator to its default, ‘.’, or ‘/’:

> mail_location = mdbox:/private/var/mail/tld.hostname.mail/%Ln/mdbox:LAYOUT=fs
> namespace inbox {
>   …
>   separator = /
>   …
> }


The IMAP client is macOS Mail.app version 14.0.

I’d greatly appreciate any help or pointers to fix this configuration issue.


dovecot -n
> # 2.3.13 (): /opt/local/etc/dovecot/dovecot.conf
> # Pigeonhole version 0.5.13 ()
> # OS: Darwin 20.4.0 x86_64  apfs
> # Hostname: mail.hostname.tld
> aps_topic = com.apple.mail.XServer.--fff-ff
> auth_cache_size = 10 M
> auth_gssapi_hostname = $ALL
> auth_krb5_keytab = /opt/local/etc/dovecot/imap.keytab
> auth_mechanisms = plain gssapi
> auth_realms =mail.hostname.tld
> auth_socket_path = /opt/local/var/run/dovecot/auth-userdb
> auth_username_format = %Ln
> debug_log_path = /opt/local/var/log/mail/mail-debug.log
> disable_plaintext_auth = no
> first_valid_gid = 6
> first_valid_uid = 6
> imap_id_log = *
> imap_id_send = "name" * "version" *
> imap_idle_notify_interval = 29 mins
> imap_urlauth_submit_user = submit
> info_log_path = /opt/local/var/log/mail/mail-info.log
> last_valid_gid = 100
> lda_mailbox_autocreate = yes
> log_path = /opt/local/var/log/mail/mail-err.log
> login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c
> mail_access_groups = mail
> mail_attachment_dir = /private/var/mail/tld.hostname.mail/attachments
> mail_attachment_fs = sis posix:mode=0666
> mail_gid = mail
> mail_home = /private/var/mail/tld.hostname.mail
> mail_location = mdbox:/private/var/mail/tld.hostname.mail/%Ln/mdbox:LAYOUT=fs
> mail_log_prefix = "%s(pid %p user %u): "
> mail_plugins = quota zlib acl fts fts_solr fts_lucene
> mail_privileged_group = mail
> mail_uid = _dovecot
> managesieve_notify_capability = mailto
> managesieve_sieve_capability = fileinto reject envelope encoded-character 
> vacation subaddress comparator-i;ascii-numeric relational regex imap4flags 
> copy include variables body enotify environment mailbox date index ihave 
> duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve
> mdbox_rotate_size = 200 M
> namespace inbox {
>   inbox = yes
>   location = 
>   mailbox Archive {
> auto = subscribe
> special_use = \Archive
>   }
>   mailbox Drafts {
> auto = subscribe
> special_use = \Drafts
>   }
>   mailbox Junk {
> auto = create
> special_use = \Junk
>   }
>   mailbox Notspam_train {
> auto = create
> special_use = \Junk
>   }
>   mailbox Sent {
> auto = subscribe
> special_use = \Sent
>   }
>   mailbox "Sent Messages" {
> special_use = \Sent
>   }
>   mailbox Spam_train {
> auto = create
> special_use = \Junk
>   }
>   mailbox Trash {
> auto = create
> special_use = \Trash
>   }
>   prefix = 
>   separator = /
> }
> passdb {
>   driver = pam
>   name = pam
> }
> plugin {
>   fts = solr
>   fts_autoindex = yes
>   fts_autoindex_exclude = \Junk
>   fts_solr = url=http://127.0.0.1:8983/solr/dovecot/
>   imapsieve_mailbox1_before = 
> file:/opt/local/etc/dovecot/sieve/report-spam.sieve
>   imapsieve_mailbox1_causes = COPY APPEND
>   imapsieve_mailbox1_name = Spam_train
>   imapsieve_mailbox2_before = 
> file:/opt/local/etc/dovecot/sieve/report-ham.sieve
>   imapsieve_mailbox2_causes = COPY APPEND
>   imapsieve_mailbox2_name = Notspam_train
>   mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename
>   mail_log_fields = uid box msgid from subject size flags
>   quota_grace = 10%%
>   quota_rule = *:storage=16G
>   quota_rule2 = Trash:storage=+256M
>   quota_warning = storage=100%% quota-exceeded %u
>   quota_warning2 = storage=80%% quota-warning %u
>   recipient_delimiter = +
>   sieve = /private/var/mail/tld.hostname.mail/rules/%Ln/dovecot.sieve
>   sieve_after = /opt/local/etc/dovecot/sieve-after.d
>   sieve_before = /opt/local/etc/dovecot/sieve-before.d
>   sieve_dir = /private/var/mail/tld.hostname.mail/rules/%Ln/%u
>   sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment
>   sieve_pipe_bin_dir = /opt/local/etc/dovecot/sieve
>   sieve_plugins = sieve_imapsieve sieve_extprograms
>   sieve_quota_max_storage = 50M
> }
> postmaster_address = postmas...@hostname.tld
> protocols = imap lmtp sieve
> quota_full_tempfail = yes
> sendmail_path = /opt/local/sbin/sendmail
> service auth-worker {
>   group = mail
>   user = root
> }
> service auth {
>   extra_groups = _keytabusers
>   idle_kill = 15 mins
>   unix_lis

Dovecot core-2.3.11.3 compile error on macOS with src/lib/ioloop-notify-kqueue.c

[Steven Smith]

I’ve hit this compile error on macOS 10.15 with the file 
src/lib/ioloop-notify-kqueue.c:

> error: implicit declaration of function 'i_gettimeofday' is invalid in C99

The fix is to #include "time-util.h"

> --- ./src/lib/ioloop-notify-kqueue.c  2020-09-18 19:56:38.0 -0400
> +++ ./src/lib/ioloop-notify-kqueue.c  2020-09-18 20:54:36.0 -0400
> @@ -11,6 +11,7 @@
>  
>  #include "ioloop-private.h"
>  #include "llist.h"
> +#include "time-util.h"
>  #include 
>  #include 
>  #include 




smime.p7s
Description: S/MIME cryptographic signature

Re: macOS Notes.app IMAP Syncing Not Working

[Steven Smith via dovecot]

Thank you very much for posting this. Looking at the diffs across 
configurations, I tried all these options, without success:

* service dns_client block
* turn off global separator variable
* auth_mechanisms login plain gssapi cram-md5

My dovecot configuration is the common denominator of multiple Notes.app 
clients not working, so I’m pretty sure I’ve configured it in a way that macOS 
Notes.app doesn’t like.

The only other difference I see is the ldap passwd and userdb driver.

Looks like I’ll have to wait for some inspiration and free time to break out 
wireshark to dig further.

Thanks again,

Steve

Re: macOS Notes.app IMAP Syncing Not Working

[Steven Smith via dovecot]

It does not appear that macOS Notes.app is authenticating to dovecot. I haven’t 
broken out wireshark yet, but I see in the dovecot logs that a working iOS 
access looks like this:

mail-debug.log (mailbox access):
> May 30 20:16:39 imap(pid 28792 user username): Debug: Mailbox Notes: Mailbox 
> open
ed because: SELECT

mail-info.log (solr indexing of a newly added Note):
> May 30 18:44:31 indexer-worker(pid 26589 user username): Info: Indexed 1 
> messages
 in Notes (UIDs 167..167)

There are no such log entries when I launch macOS Notes.app and toggle account 
activation in System Preferences>Internet Accounts.

It doesn’t look like macOS Notes.app is even trying to talk to dovecot.

Re: macOS Notes.app IMAP Syncing Not Working

[Steven Smith via dovecot]

* Ralph, Steven Smith via dovecot:

> > The issue is that macOS Note.app does not sync with this server.

> I'm syncing macOS Mojave's Notes.app with Dovecot without problems, so
> it can be done. Have you made sure that you activated both mail and
> notes in macOS' Internet Accounts preferences for this particular IMAP
> account? If so, is that IMAP account listed in the navigation bar of
> Notes.app (mine shows iCloud and the IMAP account)?

Thanks, yes, I’ve done all that and more, including:
* Setting the Path Prefix to blank, INBOX, INBOX., and ‘.’
* sqlite3 
~/Library/Containers/com.apple.Notes/Data/Library/Notes/NotesV7.storedata 
'select * from ZACCOUNT;'
* sqlite3 
~/Library/Containers/com.apple.Notes/Data/Library/Notes/NotesV7.storedata 
“update ZACCOUNT set ZUSERNAME='em...@example.com', 
ZHOSTNAME='imap.hostname.com',ZSERVERPATHPREFIX='INBOX.' where Z_PK=5;”
* Variations on the above
Reports of problems syncing Notes.app to a IMAP accounts are widespread.
I’m pretty sure it arises from some subtle dovecot configuration setting.
Would you mind posting your `doveconf -n` so that we can compare to the 
non-working and working examples above?
Steve

macOS Notes.app IMAP Syncing Not Working

[Steven Smith via dovecot]

I’m requesting help getting macOS’s Notes.app to sync notes with a new dovecot 
IMAP server.

Thanks to dovecot’s excellent online documentation and help from this list, I 
have a new perfectly functioning IMAP server.

The issue is that macOS Note.app does not sync with this server. However, iOS 
Notes.app *does* sync and work well with this IMAP server, and the differences 
between macOS and iOS Notes apps are well known. (Thanks, Apple.)
https://apple.stackexchange.com/questions/265579/notes-wont-sync-via-imap-in-macos-sierra
 

https://jpmens.net/2015/09/28/experiments-using-imap-for-notes/ 

…

I am migrating from a working Server.app system in which Notes.app syncing does 
work across all devices, and therefore have a working dovecot configuration 
from Server.app, although it does not contain an explicit configuration for the 
“Notes” mailbox.

I would greatly appreciate any help or pointers for these questions:

Does anyone have a working template for dovecot that works with macOS 
Notes.app? I’ve posted doveconf -n of both the new server and the old 
Server.app server below.

Would you all be able to provide advice on TO-DO’s and NOT-TO-DO’s for 
mailbox/name-space reconfiguration as I try to muck around with a nicely 
working, deployed IMAP server? I believe that the solution has something to do 
with name spaces and separators, and I’ve already changed the separator from 
the original ‘/’  to ‘.’ on the working server without success or obvious 
changes.

All the mail boxes on the new IMAP server look like this, *without* a prefix 
‘.’:
> ls -1 /var/mail/tld.domain.mail/username/mdbox/mailboxes/
> Archive/
> Deleted Messages/
> Drafts/
> INBOX/
> Junk/
> Notes/
> Sent/
> Trash/


All the mailboxes on the working Server.app mailserver look like this, *with* a 
prefix ‘.’:
> $ sudo ls -1 /Library/Server/Mail/Data/mail/users/username
> .Archive
> .Deleted Messages
> .Drafts
> .Junk
> .Notes
> .Sent Messages
> cur
> …
> new
> subscriptions
> tmp



Here are the `doveconf -n` of the new server that doesn’t sync macOS Notes.app, 
and the old Server.app IMAP server that does:

New server `dovecot -n` [Note: original separator was set to ‘/’, now changed 
to ‘.’ in config.d/10-mail.conf]:
> # 2.3.6 (): /opt/local/etc/dovecot/dovecot.conf
> # Pigeonhole version 0.5.6 ()
> # OS: Darwin 18.6.0 x86_64  apfs
> # Hostname: newhost.domain.tld
> auth_cache_size = 10 M
> auth_gssapi_hostname = $ALL
> auth_krb5_keytab = /opt/local/etc/dovecot/imap.keytab
> auth_mechanisms = plain gssapi
> auth_realms = newhost.domain.tld
> auth_socket_path = /opt/local/var/run/dovecot/auth-userdb
> auth_username_format = %Ln
> debug_log_path = /opt/local/var/log/mail/mail-debug.log
> disable_plaintext_auth = no
> first_valid_gid = 6
> first_valid_uid = 6
> imap_id_log = *
> imap_id_send = "name" * "version" *
> imap_idle_notify_interval = 29 mins
> imap_urlauth_submit_user = submit
> info_log_path = /opt/local/var/log/mail/mail-info.log
> last_valid_gid = 100
> lda_mailbox_autocreate = yes
> log_path = /opt/local/var/log/mail/mail-err.log
> login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c
> mail_access_groups = mail
> mail_attachment_dir = /private/var/mail/tld.domain.mail/attachments
> mail_attachment_fs = sis posix:mode=0666
> mail_debug = yes
> mail_gid = mail
> mail_home = /private/var/mail/tld.domain.mail
> mail_location = mdbox:/private/var/mail/tld.domain.mail/%Ln/mdbox
> mail_log_prefix = "%s(pid %p user %u): "
> mail_plugins = quota zlib acl fts fts_solr fts_lucene
> mail_privileged_group = mail
> mail_uid = _dovecot
> managesieve_notify_capability = mailto
> managesieve_sieve_capability = fileinto reject envelope encoded-character 
> vacation subaddress comparator-i;ascii-numeric relational regex imap4flags 
> copy include variables body enotify environment mailbox date index ihave 
> duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve
> mdbox_rotate_size = 200 M
> namespace inbox {
>   inbox = yes
>   location = 
>   mailbox Archive {
> auto = subscribe
> special_use = \Archive
>   }
>   mailbox Drafts {
> auto = subscribe
> special_use = \Drafts
>   }
>   mailbox Junk {
> auto = create
> special_use = \Junk
>   }
>   mailbox Notspam_train {
> auto = create
> special_use = \Junk
>   }
>   mailbox Sent {
> auto = subscribe
> special_use = \Sent
>   }
>   mailbox "Sent Messages" {
> special_use = \Sent
>   }
>   mailbox Spam_train {
> auto = create
> special_use = \Junk
>   }
>   mailbox Trash {
> auto = create
> special_use = \Trash
>   }
>   prefix = 
>   separator = .
> }
> passdb {
>   driver = pam
>   name = pam
> }
> plugin {
>   fts = solr
>   fts_autoindex = yes
>   fts_autoindex_exclude = \Junk
>   fts_solr = url=http://127.0.0.1:8983/solr/dovecot/
>   imap

Re: imap userdb Fatal setuid errors

[Steven Smith via dovecot]

Thank you very much! I followed your advice, upgraded to the latest release, 
and see that this issue is fixed. The new dovecot server is running perfectly.

Steve

> On May 27, 2019, at 00:37, Aki Tuomi  wrote:
> 
> 
>> On 24 May 2019 17:11 Steven Smith via dovecot  wrote:
>> 
>> 
>> I’m trying to configure dovecot lmtp in multi-user mode. My error logs are 
>> filled with messages saying that an imap process cannot do a setuid to 
>> another user:
>> 
>>> May 21 22:28:46 imap(pid 17441 user myuser): Fatal: setuid(512(myuser) from 
>>> userdb lookup) failed with euid=501(adminuser): Operation not permitted 
>>> (This binary should probably be called with process user set to 512(myuser) 
>>> instead of 501(adminuser))
>> 
>> I see that others have had similar issues, but I am not able to apply any of 
>> the fixes or workarounds to solve this issue (e.g. setting 
>> libexec/dovecot/imap as setuid-root). I’ve also tried other fixes like 
>> setting the permissions to 0777 on the userdb auth for postfix smtpd.
>> 
>> According to the code (restrict-access.c, linked below), it appears that 
>> when a user authenticates, an imap worker process is launched that has the 
>> bid of the authenticator. When another user authenticates, this last process 
>> is used, but it does not have the permissions to perform a setuid to the new 
>> user, resulting in the Fatal error that appears in the logs.
>> 
>> Is this a bug, or a configuration issue? I’ve posted my doveconf below.
>> 
>> Any pointers would be greatly appreciated.
>> 
>> Steve
>> 
> 
> service imap {
>  client_limit = 16
>  process_limit = 200
>  process_min_avail = 6
>  service_count = 0
> } 
> 
> This causes the imap process to be reused, but it cannot change it's 
> personality anymore. Also we *do not* recommend this configuration at all. 
> You are putting multiple connections inside an single-threaded binary which 
> can become "stuck" for other users when single user performs long-lasting 
> operations.
> 
> Try this:
> 
> service imap {
>  process_limit = 200
>  process_min_avail = 6
> } 
> 
> Aki

imap userdb Fatal setuid errors

[Steven Smith via dovecot]

I’m trying to configure dovecot lmtp in multi-user mode. My error logs are 
filled with messages saying that an imap process cannot do a setuid to another 
user:

> May 21 22:28:46 imap(pid 17441 user myuser): Fatal: setuid(512(myuser) from 
> userdb lookup) failed with euid=501(adminuser): Operation not permitted (This 
> binary should probably be called with process user set to 512(myuser) instead 
> of 501(adminuser))


I see that others have had similar issues, but I am not able to apply any of 
the fixes or workarounds to solve this issue (e.g. setting libexec/dovecot/imap 
as setuid-root). I’ve also tried other fixes like setting the permissions to 
0777 on the userdb auth for postfix smtpd.

According to the code (restrict-access.c, linked below), it appears that when a 
user authenticates, an imap worker process is launched that has the bid of the 
authenticator. When another user authenticates, this last process is used, but 
it does not have the permissions to perform a setuid to the new user, resulting 
in the Fatal error that appears in the logs.

Is this a bug, or a configuration issue? I’ve posted my doveconf below.

Any pointers would be greatly appreciated.

Steve


Related:
https://serverfault.com/questions/930245/dovecot-operation-not-permitted 

https://dovecot.org/list/dovecot/2012-May/135549.html 

https://trac.macports.org/ticket/58506 

Code at https://github.com/dovecot/core/blob/master/src/lib/restrict-access.c 
:
https://github.com/dovecot/core/blob/863887d4272f962926ab279ac4cf37855dd2008d/src/lib/restrict-access.c#L238-L256
 

https://github.com/dovecot/core/blob/863887d4272f962926ab279ac4cf37855dd2008d/src/lib/restrict-access.c#L342-L346
 


doveconf -n:
> # 2.3.0.1 (ffd8a29): /opt/local/etc/dovecot/dovecot.conf
> # Pigeonhole version 0.5.0.1 (d33dca20)
> # OS: Darwin 18.6.0 x86_64  apfs
> auth_cache_size = 10 M
> auth_gssapi_hostname = $ALL
> auth_krb5_keytab = /opt/local/etc/dovecot/imap.keytab
> auth_mechanisms = plain gssapi
> auth_realms = host.domain.tld
> auth_socket_path = /opt/local/var/run/dovecot/auth-userdb
> auth_username_format = %Ln
> debug_log_path = /opt/local/var/log/mail/mail-debug.log
> default_internal_user = _dovecot
> default_login_user = _dovenull
> disable_plaintext_auth = no
> first_valid_gid = 6
> first_valid_uid = 6
> imap_id_log = *
> imap_id_send = "name" * "version" *
> imap_idle_notify_interval = 29 mins
> imap_urlauth_submit_user = submit
> info_log_path = /opt/local/var/log/mail/mail-info.log
> last_valid_gid = 100
> lda_mailbox_autocreate = yes
> log_path = /opt/local/var/log/mail/mail-err.log
> login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c
> mail_access_groups = mail
> mail_attachment_dir = /private/var/mail/tld.domain.mail/attachments
> mail_attachment_fs = sis posix:mode=0666
> mail_debug = yes
> mail_gid = mail
> mail_home = /private/var/mail/tld.domain.mail
> mail_location = mdbox:/private/var/mail/tld.domain.mail/%Ln/mdbox
> mail_log_prefix = "%s(pid %p user %u): "
> mail_plugins = quota zlib acl fts fts_solr fts_lucene
> mail_privileged_group = mail
> mail_uid = _dovecot
> managesieve_notify_capability = mailto
> managesieve_sieve_capability = fileinto reject envelope encoded-character 
> vacation subaddress comparator-i;ascii-numeric relational regex imap4flags 
> copy include variables body enotify environment mailbox date index ihave 
> duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve
> mdbox_rotate_size = 200 M
> namespace inbox {
>   inbox = yes
>   location = 
>   mailbox Archive {
> auto = subscribe
> special_use = \Archive
>   }
>   mailbox Drafts {
> auto = subscribe
> special_use = \Drafts
>   }
>   mailbox Junk {
> auto = create
> special_use = \Junk
>   }
>   mailbox Notspam_train {
> auto = create
> special_use = \Junk
>   }
>   mailbox Sent {
> auto = subscribe
> special_use = \Sent
>   }
>   mailbox Spam_train {
> auto = create
> special_use = \Junk
>   }
>   mailbox Trash {
> auto = create
> special_use = \Trash
>   }
>   prefix = 
>   separator = /
> }
> passdb {
>   driver = pam
>   name = pam
> }
> plugin {
>   fts = solr
>   fts_autoindex = yes
>   fts_autoindex_exclude = \Junk
>   fts_solr = url=http://127.0.0.1:8983/solr/dovecot/
>   imapsieve_mailbox1_before = 
> file:/opt/local/etc/dovecot/sieve/report-spam.sieve
>   imapsieve_mailbox1_causes = COPY APPEND
>   imapsieve_mailbox1_name = Spam_train
>   imapsieve_mailbox2_before = 
> file:/opt/local/et