"panic ... assertion failed" with virtual and flatcurve fts together
Hi, I'm getting the following error: $ doveadm fetch 'mailbox storageid' guid '1721115116.M995635P1988.exoco.falsifian.org,S=28763,W=29300' Error: net_connect_unix(/var/dovecot/stats-writer) failed: Permission denied doveadm(falsifian): Panic: file index-search-result.c: line 174 (index_search_result_update_appends): assertion failed: (result->search_args->args == _arg) When I remove "virtual" from my "plugins" configuration line and remove the virtual namespace, the search works: $ doveadm fetch 'mailbox storageid' guid '1721115116.M995635P1988.exoco.falsifian.org,S=28763,W=29300' Error: net_connect_unix(/var/dovecot/stats-writer) failed: Permission denied mailbox: Spam storageid: 1721115116.M995635P1988.exoco.falsifian.org,S=28763,W=29300 It also works if I remove fts and fts_flatcurve from my "plugins" configuration line, leaving "virtual" alone. I first saw this when I enabled fts (with fts_flatcurve): I saw a lot of such errors in the logs when I attempted full text searches via neomutt. The errors seemed to stop happening after the indexing jobs finally finished, but just today I saw it again with the above query unrelated to fts. I have been using virtual mailboxes for a long time and hadn't noticed the error before trying fts. $ dovecot --version 2.3.21 (47349e2482) Output of doveconf -n (full config that triggers the error) follows signature. I saw something similar was reported in January [0] (that gave me the idea to try turning off virtual). Please let me know what I can do to help debug. It's just my personal mail server so I might be able to do things like run a debug build if necessary (assuming I find the time). [0] https://dovecot.org/mailman3/archives/list/dovecot@dovecot.org/thread/Z2HTMX464UUX7ESFTJVPPS7GF7IMUJQK/ -- James # 2.3.21 (47349e2482): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.21 (f6cd4b8e) # OS: OpenBSD 7.5 amd64 ffs # Hostname: exoco.falsifian.org first_valid_uid = 1000 mail_location = maildir:/home/%u/Maildir mail_plugins = " fts fts_flatcurve virtual" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext mbox_write_locks = fcntl mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / } namespace virtual { location = virtual:/mail/%u/virtual prefix = virtual/ separator = / } passdb { args = /etc/dovecot/passdb driver = passwd-file } plugin { fts = flatcurve fts_filters = lowercase fts_flatcurve_substring_search = yes fts_languages = en fts_tokenizers = generic email-address } protocols = imap ssl_cert =
Re: dovecot replication
On Fri, Jul 12, 2024 at 06:28:13PM GMT, John Fawcett via dovecot wrote: Hi James I want to avoid the -1 parameter because it doesn't do deletes in the target. -l, not -1. As for the lda to doveadm sync script, I have been wondering too about how to close the gap for emails arriving between syncs, even though the risk might not be so significant. With delivering to two dovecot servers before accepting the email, either one going down will stop email delivery. I was thinking my script will accept the email anyway if the sync fails. It would do this: 1. Pass to dovecot-lda. If dovecot-lda fails, something is seriously wrong, so stop and fail. 2. Fork a background process that attempts to doveadm sync. 3. Wait for the background process to finish, or a maximum of 2 seconds. Then return success regardless of whether sync worked. This guards against a hard disk crash or filesystem failure on one machine, but falls back to single-homing the email if a server is down. This is inspired by the documentation at https://doc.dovecot.org/configuration_manual/replication/ -- James ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: dovecot replication
Replication is in the current dovecot version but will go away in 2.4. The doveadm sync feature is staying. So with some work you can set it up what you are requesting. I used to use replication and now I'm thinking about using sync but have not implemented it. The following are thoughs on it. There are some points to be addressed that are outside dovecot. I think you'd have to make sure that your sync happened frequently enough that you could live with losing the emails that arrives bewteen syncs for example. I have been thinking of writing a hacky delivery script that passes the email on to dovecot-lda, then runs doveadm sync, and only returns success after the sync is done (or after a timeout). No idea what problems I will run into, but the idea is to never accept an email until it's guaranteed replicated. That would tend to lead to a requirement to sync more frequently and reduce risk of email loss. But then you'd need to avoid more than one sync being active simultaneously (that is my assumption that this would not work, but I don't know if it is a real problem). Doesn't the -l option protect against simultaneous syncs? Just based on reading the doveadm-sync man page. (I guess it could cause a problem if you start sync processes more quickly than they can finish.) NB I'm just running a one-person email server so don't take my ideas too seriously :) -- James ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: dovecot replication problem
Hi, I am experiencing similar error messages when setting passdb to mechanisms = plain login. More details can be found at https://github.com/orgs/docker-mailserver/discussions/3836. I hope this information helps. Regards, James On 5/07/2024 02:32, Oliver Krone via dovecot wrote: Hi there, I'm experiencing the following when I replicate mails between two host, using different versions of dovecot: *Host A Host B result* dovecot-2.3-19 --> dovecort-2.3.16 ERROR: auth: Error: All password databases were skipped. doveadm(10.0.0.1,oliver.kr...@snoog.ch): Error: auth-master: passdb lookup(oliver.kr...@snoog.ch): Auth PASS lookup failed mail | 2024-07-04T14:21:35.635047+00:00 mail dovecot: doveadm(10.0.0.1,oliver.kr...@snoog.ch): Error: /run/dovecot/auth-userdb: passdb lookup failed (to see if user is proxied, because doveadm_port is set) dovecot-2.3-21 --> dovecort-2.3.16 no errors, working Basically it does work with 2.3-21 --> 2.3-16 and it does not work with 2.3-19 ---> 2.3-16 without any changes in the configuration files. Any ideas? Thank you very much. Best regards Oliver ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: Configure: "Can't build with MySQL support: libmysqlclient not found"
On 13 Mar 2024, at 12:14 AM, Aki Tuomi wrote: On 12/03/2024 14:46 EET James L. Brown via dovecot wrote: On 12 Mar 2024, at 10:06 PM, Odhiambo Washington wrote: On Tue, Mar 12, 2024 at 4:20 AM James L. Brown via dovecot mailto:dovecot@dovecot.org>> wrote: Hi all. I’m getting errors when trying to configure make for version 2.3.21 on macOS 14.3.1 Sonoma. It ends with: checking for mysql_config... mysql_config checking for mysql_init in -lmysqlclient... no configure: error: Can't build with MySQL support: libmysqlclient not found I’m using: env PKG_CONFIG_PATH=/opt/homebrew/lib/pkgconfig pkg-config --libs libsodium CPPFLAGS=-I/opt/homebrew/Cellar/openssl@3/3.2.1/include LDFLAGS=-L/opt/ homebrew/opt/openssl@3/lib ./configure --with-ssl=openssl - -with-mysql --with- sodium Looking at config.log I see: configure:28222: checking for mysql_config configure:28238: found /opt/homebrew/bin/mysql_config configure:28250: result: mysql_config configure:28285: checking for mysql_init in -lmysqlclient configure:28310: gcc -o conftest -std=gnu99 -g -O2 -fstack- protector-strong - U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Wall -W -Wmissing- prototypes -Wmissing- declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 - Wbad-function-cast - Wno-duplicate-decl-specifier -Wstrict-aliasing=2 -I/opt/ homebrew/Cellar/ openssl@3/3.2.1/include -I/opt/homebrew/Cellar/openssl@3/ 3.2.1/include -L/opt/ homebrew/opt/openssl@3/lib conftest.c -lmysqlclient -L/ opt/homebrew/Cellar/ mysql/8.3.0_1/lib -lmysqlclient -lz -lzstd -lssl - lcrypto -lresolv -lz -lm &5 ld: warning: ignoring duplicate libraries: '-lmysqlclient', '-lz' ld: library 'zstd' not found clang: error: linker command failed with exit code 1 (use - v to see invocation) You are missing zstd library. Aki Thanks Aki. zstd 1.5.5 is installed with the other Homebriew-installed stuff at /opt/ homebrew/opt/ Not sure why it is not finding zest or libmysqlclient. I’ve even just added /opt/homebrew/opt to my PATH: % echo $PATH /opt/homebrew/opt/openssl@3/bin:/opt/homebrew/sbin:/opt/homebrew/bin:/opt/ homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/ usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/ bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/ bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/ usr/appleinternal/bin:/Library/Apple/usr/bin:/opt/homebrew/opt James. ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: Configure: "Can't build with MySQL support: libmysqlclient not found"
On 12 Mar 2024, at 10:06 PM, Odhiambo Washington wrote: On Tue, Mar 12, 2024 at 4:20 AM James L. Brown via dovecot wrote: Hi all. I’m getting errors when trying to configure make for version 2.3.21 on macOS 14.3.1 Sonoma. It ends with: checking for mysql_config... mysql_config checking for mysql_init in -lmysqlclient... no configure: error: Can't build with MySQL support: libmysqlclient not found I’m using: env PKG_CONFIG_PATH=/opt/homebrew/lib/pkgconfig pkg-config --libs libsodium CPPFLAGS=-I/opt/homebrew/Cellar/openssl@3/3.2.1/include LDFLAGS=-L/opt/ homebrew/opt/openssl@3/lib ./configure --with-ssl=openssl - -with-mysql --with- sodium Looking at config.log I see: configure:28222: checking for mysql_config configure:28238: found /opt/homebrew/bin/mysql_config configure:28250: result: mysql_config configure:28285: checking for mysql_init in -lmysqlclient configure:28310: gcc -o conftest -std=gnu99 -g -O2 -fstack- protector-strong - U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Wall -W -Wmissing- prototypes -Wmissing- declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 - Wbad-function-cast - Wno-duplicate-decl-specifier -Wstrict-aliasing=2 -I/opt/ homebrew/Cellar/ openssl@3/3.2.1/include -I/opt/homebrew/Cellar/openssl@3/ 3.2.1/include -L/opt/ homebrew/opt/openssl@3/lib conftest.c -lmysqlclient -L/ opt/homebrew/Cellar/ mysql/8.3.0_1/lib -lmysqlclient -lz -lzstd -lssl - lcrypto -lresolv -lz -lm >&5 ld: warning: ignoring duplicate libraries: '-lmysqlclient', '-lz' ld: library 'zstd' not found clang: error: linker command failed with exit code 1 (use - v to see invocation) configure:28310: $? =1 Odd that it is using the old version mysql. Would someone be able to help me get it to find libmysqlclient? In /opt/homebrew/Cellar/mysql/8.0.32/lib/ I have: libmysqlclient.21.dylib libmysqlclient.a libmysqlclient.dylib Thanks, James. Maybe https://github.com/PyMySQL/mysqlclient/issues/496#issuecomment- 1614688099 -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 In an Internet failure case, the #1 suspect is a constant: DNS. "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart- questions.html] Thanks Odhiambo, that looked promising. I used: export MYSQLCLIENT_LDFLAGS="-L/opt/homebrew/opt/mysql/lib -lmysqlclient -rpath /opt/homebrew/opt/mysql/lib" Before the env and CPPFLAGS lines. But I still get: checking for mysql_init in -lmysqlclient... no configure: error: Can't build with MySQL support: libmysqlclient not found And config.log still has in the ‘Output variables’ section: MYSQL_CONFIG='mysql_config' MYSQL_LIBS='-L/opt/homebrew/Cellar/mysql/8.3.0_1/lib -lmysqlclient -lz -lzstd -lssl -lcrypto -lresolv' I’m sure it’s something like what you suggested. Thanks, James. ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Configure: "Can't build with MySQL support: libmysqlclient not found"
Hi all. I’m getting errors when trying to configure make for version 2.3.21 on macOS 14.3.1 Sonoma. It ends with: checking for mysql_config... mysql_config checking for mysql_init in -lmysqlclient... no configure: error: Can't build with MySQL support: libmysqlclient not found I’m using: env PKG_CONFIG_PATH=/opt/homebrew/lib/pkgconfig pkg-config --libs libsodium CPPFLAGS=-I/opt/homebrew/Cellar/openssl@3/3.2.1/include LDFLAGS=-L/opt/ homebrew/opt/openssl@3/lib ./configure --with-ssl=openssl --with-mysql --with- sodium Looking at config.log I see: configure:28222: checking for mysql_config configure:28238: found /opt/homebrew/bin/mysql_config configure:28250: result: mysql_config configure:28285: checking for mysql_init in -lmysqlclient configure:28310: gcc -o conftest -std=gnu99 -g -O2 -fstack-protector-strong - U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Wall -W -Wmissing-prototypes -Wmissing- declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast - Wno-duplicate-decl-specifier -Wstrict-aliasing=2 -I/opt/homebrew/Cellar/ openssl@3/3.2.1/include -I/opt/homebrew/Cellar/openssl@3/3.2.1/include -L/opt/ homebrew/opt/openssl@3/lib conftest.c -lmysqlclient -L/opt/homebrew/Cellar/ mysql/8.3.0_1/lib -lmysqlclient -lz -lzstd -lssl -lcrypto -lresolv -lz -lm >&5 ld: warning: ignoring duplicate libraries: '-lmysqlclient', '-lz' ld: library 'zstd' not found clang: error: linker command failed with exit code 1 (use -v to see invocation) configure:28310: $? =1 Odd that it is using the old version mysql. Would someone be able to help me get it to find libmysqlclient? In /opt/homebrew/Cellar/mysql/8.0.32/lib/ I have: libmysqlclient.21.dylib libmysqlclient.a libmysqlclient.dylib Thanks, James. ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Dbox-storage error when compiling: no member named 'st_atim' in 'struct stat'
Hi all. I’m getting errors when trying to compile 2.3.21 on macOS 14.3.1 Sonoma. It ends with: 3.2.1/include -MT dbox-storage.lo -MD -MP -MF .deps/dbox-storage.Tpo -c dbox- storage.c -fno-common -DPIC -o .libs/dbox-storage.o dbox-storage.c:296:32:error:no member named 'st_atim' in 'struct stat' last_temp_file_scan = stats.st_atim.tv_sec; ~ ^ dbox-storage.c:297:24:error:no member named 'st_ctim' in 'struct stat' change_time = stats.st_ctim.tv_sec; ~ ^ 2 errors generated. make[5]: *** [dbox-storage.lo] Error 1 make[4]: *** [all-recursive] Error 1 make[3]: *** [all-recursive] Error 1 make[2]: *** [all-recursive] Error 1 make[1]: *** [all-recursive] Error 1 Before ‘make’ I ran: sudo CPPFLAGS=-I/opt/homebrew/Cellar/openssl@3/3.2.1/include LDFLAGS=-L/opt/ homebrew/opt/openssl@3/lib LDFLAGS=-L/opt/homebrew/lib ./configure --with- ssl=openssl --with-mysql --with-sodium I had to run ’sudo’ before ‘make’ as I was getting this error: error:error opening '.deps/fuzzer.Tpo': /Users/jlbrown/Downloads/dovecot- 2.3.21/src/lib-test/.deps/fuzzer.Tpo: Permission denied Would someone be able to help we with the dbox-storage error? Thanks, James. ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
doveadm sync failed after adding "mechanisms = plain login" to passdb
Hello, After adding `mechanisms = plain login` to passdb (passwd-file), `doveadm -D sync -u e...@example.com -d -N -l 30 -U` failed with ``` Jan 26 06:49:22 doveadm(e...@example.com): Error: remote(server2.example.com:61526): doveadm(1.1.1.1,e...@example.com): auth-master: passdb lookup(e...@example.com): Auth PASS lookup failed Jan 26 06:49:22 doveadm(e...@example.com): Error: remote(server2.example.com:61526): doveadm(1.1.1.1,e...@example.com): /run/dovecot/auth-userdb: passdb lookup failed (to see if user is proxied, because doveadm_port is set) ``` With `auth_debug=yes`, I found these lines. ``` Jan 26 04:55:37 server2 dovecot: auth: Debug: skipping passdb: mechanism filtered Jan 26 04:55:37 server2 dovecot: auth: Error: All password databases were skipped ``` What should I do to fix it without deleting `mechanisms = plain login`? More information could be found on [this link](https://github.com/orgs/docker-mailserver/discussions/3836#discussioncomment-8262101). Thanks ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: Setting up a new server
On 28/10/2023 07:31, James Cloos wrote: Also I'd like to use imap-specific passwds for each user. My tests so far have used the login passwds for each user. Not necessarily virtual users, just imap-specific passwds. There is already a pgsql server handy; I take it that would be the way to go for passdb and userdb lookups, yes? Yes! If you have the DB already you have done most of the work. I use postgresql for dovecot. I would urge you to use virtual users with the user name of the email address. All my users are uid:gid vmail:vmail. My mail database is used for other functions as well as dovecot but this cut down listing has the columns used by dovecot (excepting typos... keep asking). I have a table for 'mailbox'. It references table 'domain' but as it uses a text key the mailbox table will stand alone and no join on lookup. mail=# \d mailbox Table "public.mailbox" Column| Type | Collation | Nullable | Default -++---+--+- username| character varying(255) | | not null | allow_nets | character varying(255) | | | imap| boolean| | | password| character varying(255) | | | pop3| boolean| | | maxstorage | integer| | | realname| character varying(255) | | | sieve | boolean| | | smtp| boolean| | | domain | character varying(255) | | not null | maxcount| integer| | | Indexes: "mailbox_pkey" PRIMARY KEY, btree (username, domain) Foreign-key constraints: "fk_mailbox_domain" FOREIGN KEY (domain) REFERENCES domain(name) /etc/opt/.../dovecot-sql.conf has lines: user_query = "SELECT 'vmail' AS uid, 'vmail' AS gid, allow_nets, '*:storage=' || maxstorage || 'M' AS quota_rule, '*:messages=' || maxcount AS quota_rule2 FROM mailbox WHERE username = '%n' AND domain = '%d' AND smtp = true;" password_query = "SELECT password, allow_nets, '*:storage=' || maxstorage || 'M' AS userdb_quota_rule, '*:messages=' || maxcount AS userdb_quota_rule2 FROM mailbox WHERE username = '%n' AND domain = '%d' AND %Ls = true;" Take this as hints; consult the documentation. James. ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Setting up a new server
I'm in the process of switching to dovecot for my lan imap and have a couple of questions. I want to keep the folder names unchanged. For mailing lists I use the posting address (eg f...@bar.co); for others I use / as the separator. If I use mdbox, will those choices cause any issues? My impression is not, but I'd like to be sure. Also I'd like to use imap-specific passwds for each user. My tests so far have used the login passwds for each user. Not necessarily virtual users, just imap-specific passwds. There is already a pgsql server handy; I take it that would be the way to go for passdb and userdb lookups, yes? Thanks, -JimC -- James Cloos OpenPGP: 0x997A9F17ED7DAEA6 ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Bug - doveadm backup out-of-memory kill/crash when no separators set
Hi, I hit a fun issue with doveadm when migrating from dovecot 2.2.36 (1f10bfa63) to 2.3.19.1 (9b53102964) (CentOS 7 to Debian 12). When running doveadm -v -D backup -R -u "user@name" tcp:localhost:1234, I found that the first sync would always work, but subsequent runs of the command would cause doveadm to reach a subfolder (Archives/2008 in the example below) and then silently mmap() increasing powers of 2 before the OOM killer finally got it. Deleting the source folder caused it to remain stuck on the same folder, but for the deletion event. I switched memory overcommit off to force a crash, and got a gdb backtrace: #0 __pthread_kill_implementation (threadid=, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44 #1 0x7786bd9f in __pthread_kill_internal (signo=6, threadid=) at ./nptl/pthread_kill.c:78 #2 0x7781cf32 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26 #3 0x77807472 in __GI_abort () at ./stdlib/abort.c:79 #4 0x77b4ffae in default_fatal_finish (status=0, type=LOG_TYPE_PANIC) at ../lib/failures.c:465 #5 fatal_handler_real (ctx=, format=, args=) at ../lib/failures.c:477 #6 0x77bfa081 in default_fatal_handler (ctx=, format=, args=) at ../lib/failures.c:485 #7 0x77b5017c in i_panic (format=0x77c5d748 "data stack: Out of memory when allocating %zu bytes") at ../lib/failures.c:530 #8 0x77b4f67f in mem_block_alloc (min_size=min_size@entry=16) at ../lib/data-stack.c:386 #9 0x77bf8b60 in t_malloc_real (size=, permanent=) at ../lib/data-stack.c:492 #10 0x77c348f1 in t_strdup_until (start=start@entry=0x5565f440, end=end@entry=0x5565f448) at ../lib/strfuncs.c:270 #11 0x555adc62 in convert_name_to_remote_sep (name=0x5565f440 "Archives/2008", tree=0x5564e188) at dsync/dsync-mailbox-tree.c:270 #12 dsync_mailbox_tree_build_name128_remotesep_hash (tree=0x5564e188) at dsync/dsync-mailbox-tree.c:315 #13 dsync_mailbox_tree_find_delete (tree=0x5564e188, del=0x556469f0) at dsync/dsync-mailbox-tree.c:405 #14 0x555a4195 in dsync_brain_mailbox_tree_add_delete (tree=0x5564e188, other_tree=0x5564f5f8, other_del=0x556469f0, node_r=0x7fffe350, status_r=0x7fffe348) at dsync/dsync-brain-mailbox-tree.c:504 #15 0x555a44cd in dsync_brain_recv_mailbox_tree_deletes (brain=0x5564b2d8) at dsync/dsync-brain-mailbox-tree.c:590 #16 0x555a5365 in dsync_brain_run_real (brain=brain@entry=0x5564b2d8, changed_r=changed_r@entry=0x7fffe453) at dsync/dsync-brain.c:709 #17 0x555a59f9 in dsync_brain_run (changed_r=0x7fffe453, brain=0x5564b2d8) at dsync/dsync-brain.c:752 #18 dsync_brain_run (changed_r=0x7fffe453, brain=0x5564b2d8) at dsync/dsync-brain.c:740 #19 dsync_brain_run_io (context=) at dsync/dsync-brain.c:113 #20 dsync_brain_run_io (context=0x5564b2d8) at dsync/dsync-brain.c:100 #21 0x555b23df in dsync_ibc_stream_input (ibc=0x55646720) at dsync/dsync-ibc-stream.c:232 #22 0x77c11cd9 in io_loop_call_io (io=0x556418f0) at ../lib/ioloop.c:737 #23 0x77c13aa2 in io_loop_handler_run_internal (ioloop=ioloop@entry=0x5560bf40) at ../lib/ioloop-epoll.c:222 #24 0x77c13b50 in io_loop_handler_run (ioloop=ioloop@entry=0x5560bf40) at ../lib/ioloop.c:789 #25 0x77c13d10 in io_loop_run (ioloop=0x5560bf40) at ../lib/ioloop.c:762 #26 0x5558b22e in cmd_dsync_run_remote (user=0x55637248) at ./src/doveadm/doveadm-dsync.c:543 #27 cmd_dsync_run (_ctx=0x5561f288, user=0x55637248) at ./src/doveadm/doveadm-dsync.c:750 #28 0x5558bd12 in doveadm_mail_next_user (ctx=0x5561f288, error_r=0x7fffe818) at ./src/doveadm/doveadm-mail.c:464 #29 0x5558cf45 in doveadm_mail_cmd_exec (wildcard_user=0x0, ctx=0x5561f288) at ./src/doveadm/doveadm-mail.c:659 #30 doveadm_cmd_ver2_to_mail_cmd_wrapper (cctx=0x7fffe970) at ./src/doveadm/doveadm-mail.c:988 #31 0x55597622 in doveadm_cmd_run_ver2 (argc=5, argv=0x5560ba38, cctx=cctx@entry=0x7fffe970) at ./src/doveadm/doveadm-cmd.c:465 #32 0x55597697 in doveadm_cmd_try_run_ver2 (cmd_name=out>, argc=, argv=, cctx=0x7fffe970) at ./src/doveadm/doveadm-cmd.c:363 #33 0x5557919a in main (argc=, argv=out>) at ./src/doveadm/doveadm.c:361 Neither server had a separator set in the inbox namespace. Having identified the convert_name_to_remote_sep function as the likely trigger point I set the separator to / on both ends, and doveadm backup now runs without issue. Can provide config if helpful but I think probably irrelevant to this one - the namespace config is the default other than the separator item. May be worth mentioning that having the separator set can avoid this condition in the doveadm backup docs - this took me a while to fig
Re: "sh: doveadm: command not found" with doveadm -Dv backup -R
> On 28 Feb 2023, at 5:53 pm, Aki Tuomi wrote: > >> On 28/02/2023 03:54 EET James Brown wrote: >> >> I’m moving from an old mail server to a new one. >> >> On the new one, trying to transfer mailboxes from the old machine using >> doveadm. It fails saying it can’f find doveadm: >> >> % sudo doveadm -Dv backup -R -u myu...@bordo.com.au remote:mail.bordo.com.au >> Password: >> Debug: Loading modules from directory: /usr/local/lib/dovecot/doveadm >> ... >> (r...@mail.bordo.com.au) Password: >> sh: doveadm: command not found >> Feb 28 12:42:06 dsync-local(myu...@bordo.com.au): >> Error: read(mail.bordo.com.au) failed: EOF (version not received) >> Feb 28 12:42:06 dsync-local(myu...@bordo.com.au): >> Error: Remote command returned error 127: ssh mail.bordo.com.au doveadm >> dsync-server -umyu...@bordo.com.au -U >> Feb 28 12:42:06 dsync-local(myu...@bordo.com.au): >> Debug: User session is finished >> Feb 28 12:42:06 dsync-local(myu...@bordo.com.au): >> Debug: auth-master: conn unix:/usr/local/var/run/dovecot/auth-userdb >> (uid=0): Disconnected: Connection closed (fd=9) >> >> Does this mean the remote machine can’t fine doveadm? >> >> (Both machines are running macOS) >> >> I could probably transfer the mailboxes using rsync, but I thought that >> doveadm was a better way. >> >> Thanks, >> >> James. > > Yeah, it means it can't find doveadm on the remote machine. > > Please see https://doc.dovecot.org/admin_manual/migrating_mailboxes/ > > Aki Thanks Aki, those instructions are great. All working now. Added the service doveadm and pw to old server and pw to new one and all works. And used tcp:oldmailserver:port instead of ‘remote:’ in command. So sudo doveadm -Dv backup -R -u myu...@bordo.com.au tcp:mail.bordo.com.au:12354 Simple! Thanks again, James.
"sh: doveadm: command not found" with doveadm -Dv backup -R
mailbox in list index: Mailbox not found Feb 28 12:42:02 doveadm(myu...@bordo.com.au): Debug: Namespace : Using permissions from /usr/local/virtual/bordo.com.au/myuser: mode=0700 gid=default (r...@mail.bordo.com.au) Password: sh: doveadm: command not found Feb 28 12:42:06 dsync-local(myu...@bordo.com.au): Error: read(mail.bordo.com.au) failed: EOF (version not received) Feb 28 12:42:06 dsync-local(myu...@bordo.com.au): Error: Remote command returned error 127: ssh mail.bordo.com.au doveadm dsync-server -umyu...@bordo.com.au -U Feb 28 12:42:06 dsync-local(myu...@bordo.com.au): Debug: User session is finished Feb 28 12:42:06 dsync-local(myu...@bordo.com.au): Debug: auth-master: conn unix:/usr/local/var/run/dovecot/auth-userdb (uid=0): Disconnected: Connection closed (fd=9) Does this mean the remote machine can’t fine doveadm? (Both machines are running macOS) I could probably transfer the mailboxes using rsync, but I thought that doveadm was a better way. Thanks, James.
Re: Auth-worker, unknown scheme ARGON2ID
On 22 Feb 2023, at 6:12 pm, Aki Tuomi wrote: > > You can use > > env PKG_CONFIG_PATH=/opt/homebrew/lib/ ./configure … > > Aki Aki was a tremendous help and got it working for me. For anyone else who wants to use Dovecot on macOS with the ARGON2I or ARGON2ID password schemes, these are the steps: First, need to install pkg-config using Homebrew. Then install libsodium from Homebrew. Then: env PKG_CONFIG_PATH=/opt/homebrew/lib/pkgconfig pkg-config --libs libsodium CPPFLAGS=-I/opt/homebrew/Cellar/openssl@3/3.0.8/include LDFLAGS=-L/opt/homebrew/opt/openssl@3/lib ./configure --with-ssl=openssl --with-mysql --with-sodium (Probably should have not linked to specific version of OpenSSL 3, but it works so I’m not changing it now!) make sudo make install All should work, start Dovecot with sudo /usr/local/sbin/dovecot And test with: % sudo doveadm pw -l SHA1 SSHA512 SCRAM-SHA-256 BLF-CRYPT PLAIN HMAC-MD5 OTP SHA512 SHA DES-CRYPT CRYPT SSHA MD5-CRYPT PLAIN-MD4 PLAIN-MD5 SCRAM-SHA-1 CLEAR CLEARTEXT ARGON2I ARGON2ID SSHA256 MD5 PBKDF2 SHA256 CRAM-MD5 PLAIN-TRUNC SMD5 DIGEST-MD5 LDAP-MD5 Hope that saves someone some time and frustration. Probably future me! :-) Huge thanks to Aki for his help and patience, James.
Re: Auth-worker, unknown scheme ARGON2ID
On 22 Feb 2023, at 5:53 pm, Aki Tuomi wrote: > >>>> >>>> % locate libsodium >>>> /opt/homebrew/Cellar/libsodium >>>> /opt/homebrew/Cellar/libsodium/1.0.18_1 >>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/.brew >>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/.brew/libsodium.rb >>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/AUTHORS >>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/ChangeLog >>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/INSTALL_RECEIPT.json >>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/LICENSE >>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/README.markdown >>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/include >>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium >>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium/core.h >>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium/crypto_aead_aes256gcm.h >>>> ... >>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium/utils.h >>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium/version.h >>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium.h >>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/lib >>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/lib/libsodium.23.dylib >>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/lib/libsodium.a >>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/lib/libsodium.dylib >>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/lib/pkgconfig >>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/lib/pkgconfig/libsodium.pc >>>> /opt/homebrew/lib/libsodium.23.dylib >>>> /opt/homebrew/lib/libsodium.a >>>> /opt/homebrew/lib/libsodium.dylib >>>> /opt/homebrew/lib/pkgconfig/libsodium.pc >>>> /opt/homebrew/opt/libsodium >>>> /opt/homebrew/var/homebrew/linked/libsodium >>>> >>>> What do I need to use to stop the:Can't build with libsodium: not found >>>> >>>> Sorry these are such stupid questions. >>>> >>>> James. > > Dovecot uses pkg-config to find out about libsodium, and it seems you're > missing libsodium.pc in whatever place mac uses to hide them. > > Maybe the .pc file is in the homebrew directory somewhere and needs to be > copied, or you can make it yourself. Make sure to change the version and > paths. > > prefix=/usr > exec_prefix=${prefix} > libdir=${prefix}/lib/x86_64-linux-gnu > includedir=${prefix}/include > > Name: libsodium > Version: 1.0.18 > Description: A modern and easy-to-use crypto library > > Libs: -L${libdir} -lsodium > Libs.private: -pthread > Cflags: -I${includedir} > > Aki Getting somewhere - thanks Aki. % locate libsodium.pc /opt/homebrew/Cellar/libsodium/1.0.18_1/lib/pkgconfig/libsodium.pc /opt/homebrew/lib/pkgconfig/libsodium.pc /usr/local/lib/pkgconfig/libsodium.pc Looking at /opt/homebrew/lib/pkgconfig/libsodium.pc it is: prefix=/opt/homebrew/Cellar/libsodium/1.0.18_1 exec_prefix=${prefix} libdir=${exec_prefix}/lib includedir=${prefix}/include Name: libsodium Version: 1.0.18 Description: A modern and easy-to-use crypto library Libs: -L${libdir} -lsodium Libs.private: -pthread Cflags: -I${includedir} The version of libsodium I downloaded and installed from source is: prefix=/usr/local exec_prefix=${prefix} libdir=${exec_prefix}/lib includedir=${prefix}/include Name: libsodium Version: 1.0.18 Description: A modern and easy-to-use crypto library Libs: -L${libdir} -lsodium Libs.private: -lpthread -pthread Cflags: -I${includedir} Do I need to edit one of these files, or somehow tell .configure to look at /opt/homebrew/lib/ ? James.
Re: Auth-worker, unknown scheme ARGON2ID
> On 22 Feb 2023, at 5:33 pm, Aki Tuomi wrote: >> >> Thanks Aki, that was helpful. When I add that I get: >> >> checking for LIBSODIUM... no >> configure: error: Can't build with libsodium: not found >> >> So I have to tell it where libsodium is. >> >> Tried: >> >> CPPFLAGS=-I/opt/homebrew/Cellar/openssl@3/3.0.8/include >> LDFLAGS=-L/opt/homebrew/opt/openssl@3/lib >> LIBSODIUM_LIBS=-L/opt/homebrew/Cellar/libsodium ./configure >> --with-ssl=openssl --with-mysql —with-sodium >> >> But that gives: >> >> configure: WARNING: Libtool does not cope well with whitespace in `pwd` >> checking build system type... Invalid configuration `—-with-sodium': machine >> `—-with' not recognized >> >> Location of libsodium: >> >> % locate libsodium >> /opt/homebrew/Cellar/libsodium >> /opt/homebrew/Cellar/libsodium/1.0.18_1 >> /opt/homebrew/Cellar/libsodium/1.0.18_1/.brew >> /opt/homebrew/Cellar/libsodium/1.0.18_1/.brew/libsodium.rb >> /opt/homebrew/Cellar/libsodium/1.0.18_1/AUTHORS >> /opt/homebrew/Cellar/libsodium/1.0.18_1/ChangeLog >> /opt/homebrew/Cellar/libsodium/1.0.18_1/INSTALL_RECEIPT.json >> /opt/homebrew/Cellar/libsodium/1.0.18_1/LICENSE >> /opt/homebrew/Cellar/libsodium/1.0.18_1/README.markdown >> /opt/homebrew/Cellar/libsodium/1.0.18_1/include >> /opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium >> /opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium/core.h >> /opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium/crypto_aead_aes256gcm.h >> ... >> /opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium/utils.h >> /opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium/version.h >> /opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium.h >> /opt/homebrew/Cellar/libsodium/1.0.18_1/lib >> /opt/homebrew/Cellar/libsodium/1.0.18_1/lib/libsodium.23.dylib >> /opt/homebrew/Cellar/libsodium/1.0.18_1/lib/libsodium.a >> /opt/homebrew/Cellar/libsodium/1.0.18_1/lib/libsodium.dylib >> /opt/homebrew/Cellar/libsodium/1.0.18_1/lib/pkgconfig >> /opt/homebrew/Cellar/libsodium/1.0.18_1/lib/pkgconfig/libsodium.pc >> /opt/homebrew/lib/libsodium.23.dylib >> /opt/homebrew/lib/libsodium.a >> /opt/homebrew/lib/libsodium.dylib >> /opt/homebrew/lib/pkgconfig/libsodium.pc >> /opt/homebrew/opt/libsodium >> /opt/homebrew/var/homebrew/linked/libsodium >> >> What do I need to use to stop the:Can't build with libsodium: not found >> >> Sorry these are such stupid questions. >> >> James. > > You accidentically used a UTF-8 special dash instead of two normal dashes. > > Aki > Yes, I think I picked that up. I think I might have typed it into the email and it changed it (I’ve run the configure command so many times.) CPPFLAGS=-I/opt/homebrew/Cellar/openssl@3/3.0.8/include LDFLAGS=-L/opt/homebrew/opt/openssl@3/lib LIBSODIUM_LIBS=-L/opt/homebrew/Cellar/libsodium ./configure --with-ssl=openssl --with-mysql --with-sodium Fails with: checking for mysql_ssl_set in -lmysqlclient... yes checking for LIBSODIUM... no configure: error: Can't build with libsodium: not found This is so frustrating, but I’ll be jumping for joy if we can get it to work. James.
Re: Auth-worker, unknown scheme ARGON2ID
> On 22 Feb 2023, at 4:14 pm, Aki Tuomi wrote: > >> I’ve spent ages on this and am getting really desperate! :-( >> >> CPPFLAGS=-I/opt/homebrew/Cellar/openssl@3/3.0.8/include >> LDFLAGS=-L/opt/homebrew/opt/openssl@3/lib ./configure --with-ssl=openssl >> --with-mysql >> ... >> Install prefix . : /usr/local >> File offsets ... : 64bit >> I/O polling : kqueue >> I/O notifys : kqueue >> SSL : yes (OpenSSL) >> GSSAPI . : no >> passdbs : static passwd passwd-file pam checkpassword sql >> : -shadow -bsdauth -ldap >> userdbs : static prefetch passwd passwd-file checkpassword sql >> >> sudo doveadm pw -l >> SHA1 SSHA512 SCRAM-SHA-256 BLF-CRYPT PLAIN HMAC-MD5 OTP SHA512 SHA DES-CRYPT >> CRYPT SSHA MD5-CRYPT PLAIN-MD4 PLAIN-MD5 SCRAM-SHA-1 CLEAR CLEARTEXT SSHA256 >> MD5 PBKDF2 SHA256 CRAM-MD5 PLAIN-TRUNC SMD5 DIGEST-MD5 LDAP-MD5 >> >> How do I get ARGON2I, ARGON2ID in that list? >> >> Has anybody got Dovecot to work on recent macOS with these password schemes? >> Any hints? >> >> Thanks, James. > > You need to use --with-sodium when building. > > Aki Thanks Aki, that was helpful. When I add that I get: checking for LIBSODIUM... no configure: error: Can't build with libsodium: not found So I have to tell it where libsodium is. Tried: CPPFLAGS=-I/opt/homebrew/Cellar/openssl@3/3.0.8/include LDFLAGS=-L/opt/homebrew/opt/openssl@3/lib LIBSODIUM_LIBS=-L/opt/homebrew/Cellar/libsodium ./configure --with-ssl=openssl --with-mysql —with-sodium But that gives: configure: WARNING: Libtool does not cope well with whitespace in `pwd` checking build system type... Invalid configuration `—-with-sodium': machine `—-with' not recognized Location of libsodium: % locate libsodium /opt/homebrew/Cellar/libsodium /opt/homebrew/Cellar/libsodium/1.0.18_1 /opt/homebrew/Cellar/libsodium/1.0.18_1/.brew /opt/homebrew/Cellar/libsodium/1.0.18_1/.brew/libsodium.rb /opt/homebrew/Cellar/libsodium/1.0.18_1/AUTHORS /opt/homebrew/Cellar/libsodium/1.0.18_1/ChangeLog /opt/homebrew/Cellar/libsodium/1.0.18_1/INSTALL_RECEIPT.json /opt/homebrew/Cellar/libsodium/1.0.18_1/LICENSE /opt/homebrew/Cellar/libsodium/1.0.18_1/README.markdown /opt/homebrew/Cellar/libsodium/1.0.18_1/include /opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium /opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium/core.h /opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium/crypto_aead_aes256gcm.h ... /opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium/utils.h /opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium/version.h /opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium.h /opt/homebrew/Cellar/libsodium/1.0.18_1/lib /opt/homebrew/Cellar/libsodium/1.0.18_1/lib/libsodium.23.dylib /opt/homebrew/Cellar/libsodium/1.0.18_1/lib/libsodium.a /opt/homebrew/Cellar/libsodium/1.0.18_1/lib/libsodium.dylib /opt/homebrew/Cellar/libsodium/1.0.18_1/lib/pkgconfig /opt/homebrew/Cellar/libsodium/1.0.18_1/lib/pkgconfig/libsodium.pc /opt/homebrew/lib/libsodium.23.dylib /opt/homebrew/lib/libsodium.a /opt/homebrew/lib/libsodium.dylib /opt/homebrew/lib/pkgconfig/libsodium.pc /opt/homebrew/opt/libsodium /opt/homebrew/var/homebrew/linked/libsodium What do I need to use to stop the: Can't build with libsodium: not found Sorry these are such stupid questions. James.
Re: Auth-worker, unknown scheme ARGON2ID
On 21 Feb 2023, at 10:12 pm, James Brown wrote: > > The new one has Dovecot compiled with same configure options, same > configuration files, but fails to authenticate: > > Feb 21 21:51:03 master: Info: Dovecot v2.3.20 (80a5ac675d) starting up for > imap, pop3 (core dumps disabled) > Feb 21 21:51:33 auth-worker(11701): Error: conn unix:auth-worker (uid=214): > auth-worker<1>: > sql(jlbr...@bordo.com.au,::1,): Unknown > scheme ARGON2ID > > ./configure --with-ssl=openssl --with-mysql > > Was what I used to compile. > > Using OpenSSL 3.0.8. New server is running macOS 13.2.1 Ventura. > > (Have also installed libsodium 1.0.18_1) > > Any help would be most appreciated. > > thanks, > > James. I’ve spent ages on this and am getting really desperate! :-( CPPFLAGS=-I/opt/homebrew/Cellar/openssl@3/3.0.8/include LDFLAGS=-L/opt/homebrew/opt/openssl@3/lib ./configure --with-ssl=openssl --with-mysql ... Install prefix . : /usr/local File offsets ... : 64bit I/O polling : kqueue I/O notifys : kqueue SSL : yes (OpenSSL) GSSAPI . : no passdbs : static passwd passwd-file pam checkpassword sql : -shadow -bsdauth -ldap userdbs : static prefetch passwd passwd-file checkpassword sql sudo doveadm pw -l SHA1 SSHA512 SCRAM-SHA-256 BLF-CRYPT PLAIN HMAC-MD5 OTP SHA512 SHA DES-CRYPT CRYPT SSHA MD5-CRYPT PLAIN-MD4 PLAIN-MD5 SCRAM-SHA-1 CLEAR CLEARTEXT SSHA256 MD5 PBKDF2 SHA256 CRAM-MD5 PLAIN-TRUNC SMD5 DIGEST-MD5 LDAP-MD5 How do I get ARGON2I, ARGON2ID in that list? Has anybody got Dovecot to work on recent macOS with these password schemes? Any hints? Thanks, James.
Auth-worker, unknown scheme ARGON2ID
On my current mail server I was able to upgrade everyone’s password scheme to ARGON2ID with no issues, I just had to have: service auth { vsz_limit = 0 } This was an older version of macOS than the new server I’m trying to migrate to. The new one has Dovecot compiled with same configure options, same configuration files, but fails to authenticate: Feb 21 21:51:03 master: Info: Dovecot v2.3.20 (80a5ac675d) starting up for imap, pop3 (core dumps disabled) Feb 21 21:51:33 auth-worker(11701): Error: conn unix:auth-worker (uid=214): auth-worker<1>: sql(jlbr...@bordo.com.au,::1,): Unknown scheme ARGON2ID ./configure --with-ssl=openssl --with-mysql Was what I used to compile. Using OpenSSL 3.0.8. New server is running macOS 13.2.1 Ventura. (Have also installed libsodium 1.0.18_1) Any help would be most appreciated. thanks, James.
service(imap-login): Fatal: setrlimit(RLIMIT_DATA, 268435456): Invalid argument macOS
I fixed the issue with Dovecot not starting on macOS Ventura by adding this to 10-master.conf: service log { vsz_limit = 0 } But now when I try to login via Imap on port 143 I get the same error for imap-login: Feb 20 15:55:41 imap-login: Error: Feb 20 15:55:41 service(imap-login): Fatal: setrlimit(RLIMIT_DATA, 268435456): Invalid argument Feb 20 15:55:41 imap-login: Fatal: master: service(imap-login): child 11781 returned error 89 (Fatal failure) Feb 20 15:55:41 master: Error: service(imap-login): command startup failed, throttling for 60.000 secs Feb 20 16:13:04 imap-login: Error: Feb 20 16:13:04 service(imap-login): Fatal: setrlimit(RLIMIT_DATA, 268435456): Invalid argument Feb 20 16:13:04 imap-login: Fatal: master: service(imap-login): child 20370 returned error 89 (Fatal failure) Feb 20 16:13:04 master: Error: service(imap-login): command startup failed, throttling for 60.000 secs Config: sudo /usr/local/sbin/dovecot -n # 2.3.20 (80a5ac675d): /usr/local/etc/dovecot/dovecot.conf # OS: Darwin 22.3.0 arm64 apfs # Hostname: mail-server-mac-studio.local auth_mechanisms = plain login auth_verbose = yes auth_verbose_passwords = yes default_internal_group = mail default_internal_user = _dovecot default_login_user = _dovenull first_valid_gid = 0 first_valid_uid = 102 last_valid_gid = 501 last_valid_uid = 105 log_path = /var/log/dovecot.log mail_gid = 102 mail_location = maildir:/usr/local/virtual/%d/%u mail_uid = 102 namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } protocols = imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = _postfix mode = 0660 user = _postfix } unix_listener auth-userdb { mode = 0660 user = _vmail } } service imap-login { vsz_limit = 0 } service log { vsz_limit = 0 } ssl_cert =
Re: SOLVED: Re: Dovecot does not start on MacOS 12.01, and now 13.2.1 - RLIMIT_DATA
Sorry, replying to an old thread found on mail-archive.com <http://mail-archive.com/>, so it’s not threaded email. https://www.mail-archive.com/dovecot@dovecot.org/msg84776.html I have just installed Dovecot onto Mac Studio running Ventura (macOS 13.2.1) and get the same error with Dovecot 2.3.20: service(log): Fatal: setrlimit(RLIMIT_DATA, 268435456): Invalid argument Adding the ’service log’ stanza to 10-master.conf fixed the issue. Is this still the correct way to handle it? If so it would be great to have it documented somewhere. Just lucky I found the above mail thread. Thanks, James. Hi! That solution is bit overly broad way to solve this, I would instead recommend service log { vsz_limit = 0 } so that only service log is affected. Aki > On 31/10/2021 20:36 Don Feliciano wrote: > > > > A kind person who isn’t on the mailing list share the solution to this with > me: > > default_vsz_limit = 0 > > > > On Oct 28, 2021, at 12:12 PM, Don Feliciano > > wrote: > > > > > > Warning: fd limit (ulimit -n) is lower than required under max. load (256 < > > 1000), because of default_client_limit > > Oct 28 12:08:48 service(log): Fatal: setrlimit(RLIMIT_DATA, 268435456): > > Invalid argument > > > > > > > On Oct 27, 2021, at 11:08 AM, Don Feliciano > > > wrote: > > > > > > I've been happily running Dovecot on my Mac for many years (installed via > > > Homebrew). After upgrading to Monterey (MacOS 12.01), it no longer starts: > > > > > > $ sw_vers > > > ProductName: macOS > > > ProductVersion: 12.0.1 > > > BuildVersion: 21A559 > > > > > > $ uname -a > > > Darwin dfelicia-mac 21.1.0 Darwin Kernel Version 21.1.0: Wed Oct 13 > > > 17:33:23 PDT 2021; root:xnu-8019.41.5~1/RELEASE_X86_64 x86_64 > > > > > > $ sudo dovecot -F > > > Oct 27 10:11:18 service(log): Fatal: setrlimit(RLIMIT_DATA, 268435456): > > > Invalid argument
Permissions issue with doveadm mailbox command
I have been exploring dovecot and came across this: $ sudo doveadm mailbox list doveadm(root): Error: Mail access for users with UID 303 not permitted (see first_valid_uid in config file, uid from mail_uid setting). Hmm. After setting "first_valid_uid = 300": $ sudo doveadm mailbox list doveadm(root)<4380><4VP8EENer2McEQAAvbJltg>: Error: chdir(/root/) failed: Permission denied (euid=303(vmail) egid=303(vmail) missing +x perm: /root, dir owned by 0:0 mode=0700) INBOX I do not understand the error message. The current configuration. (Note I have changed first_valid_uid to its original value.) $ postconf -n # 2.3.15 (0503334ab1): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.15 (e6a84e31) # OS: Linux 5.14.21-150400.24.33-default x86_64 btrfs # Hostname: sma-server3.sma.com auth_mechanisms = plain login base_dir = /var/run/dovecot/ debug_log_path = /data01/var/log/dovecot/dovecot-debug.log disable_plaintext_auth = no first_valid_uid = 1001 info_log_path = /data01/var/log/dovecot/dovecot-info.log log_path = /data01/var/log/dovecot/dovecot.log log_timestamp = %Y-%m-%dT%H:%M:%S mail_gid = vmail mail_location = maildir:/var/mail/vmail/%u/ mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve } protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service log { group = users user = dovecot } ssl = no ssl_cipher_list = ALL:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PSK:!RC4:!ADH:!LOW@STRENGTH ssl_options = no_compression ssl_prefer_server_ciphers = yes userdb { driver = passwd } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } -- James Moe moe dot james at sohnen-moe dot com 520.743.3936 Think.
Re: [SOLVED] Permissions for dovecot logging
On 2022-12-27 16:19, James Moe wrote: > I changed logging to use a path rather than syslog. Doing so makes it easier > to work with fail2ban. > Dovecot fails to start with the error: > Can't open log file /data01/var/log/dovecot.log: Permission denied > Yes, it was apparmor. It has been enabled for a couple of months. Dovecot is the first app that I've added that has a apparmor profile. After adding the necessary entry to the profile, logging proceeded as expected. In I added: owner /data01/var/log/dovecot/* a, -- James Moe moe dot james at sohnen-moe dot com 520.743.3936 Think.
Re: Permissions for dovecot logging
On 2022-12-28 00:27, Aki Tuomi wrote: > The `log` service runs by default as root, not as dovecot. > Then I do not understand why there is a permissions problem at all. It is root! > If data01 is a NFS mount, then root may become squashed. > Not an NFS mount. It is local. > If you want to run log as `dovecot`, you can do so with > > service log { > user = dovecot > } > Permission is still denied. Where do I find information about "status=80/n/a"? I did not include all two of the syslog entries in the previous message: 2022-12-29T20:17:56-0700 sma-server3 dovecot[12102]: Can't open log file /data01/var/log/dovecot.log: Permission denied 2022-12-29T20:17:56-0700 sma-server3 systemd[1]: dovecot.service: Main process exited, code=exited, status=80/n/a -- James Moe moe dot james at sohnen-moe dot com 520.743.3936 Think.
Permissions for dovecot logging
dovecot 2.3.15 opensuse LEAP 15.4 I changed logging to use a path rather than syslog. Doing so makes it easier to work with fail2ban. Dovecot fails to start with the error: Can't open log file /data01/var/log/dovecot.log: Permission denied Permissions: drwxrwxr-x 1 root users 104 Feb 25 2018 /data01/ drwxrwxr-x 1 sma-user3x users 102 Dec 17 14:50 /data01/var/ drwxrwxr-x 1 sma-user3x users 146 Dec 27 15:37 /data01/var/log/ drwxrwxr-x 1 dovecotusers 22 Dec 27 15:47 /data01/var/log/dovecot/ "dovecot" is a member of "users". What "permission" am I missing? Note: A long time ago I had a problem with programs consuming all available space on the system disk with log or backup files. I have since gotten in the habit of putting log files on a non-system disk. -- James Moe moe dot james at sohnen-moe dot com 520.743.3936 Think.
Re: What do I do with the ".ext?"
On 2022-12-17 00:09, Aki Tuomi wrote: > Please see > https://doc.dovecot.org/configuration_manual/authentication/sql/#authentication-sql > how to use these. > Thank you! -- James Moe moe dot james at sohnen-moe dot com 520.743.3936 Think.
What do I do with the ".ext?"
dovecot 2.3.15 opensuse LEAP 15.4 I am installing dovecot and have postfix as the SMTP agent. It also needs to use a MySQL database. In there is the main dovecot.conf file, and a number of other files with the extension ".conf.ext." It is not clear how to use these. I could nothing in the documentation that discusses it. Do I copy one, say dovecot-sql.conf.ext, modify it: - rename the copy without the ".ext?" - or move the copy to the "conf.d" directory, with or without the "ext?" -- James Moe moe dot james at sohnen-moe dot com 520.743.3936 Think.
Re: SQL user_query quota_rule for *:messages=
On 23/05/2022 08:18, Aki Tuomi wrote: Is it possible to override the message count limit with an SQL user_query (and password_query)? I tried returning userdb_quota_rule2 in the SQL but it did not work. "SELECT ... '*:messages=' || maxcount AS userdb_quota_rule2 ...;" Ref: https://doc.dovecot.org/configuration_manual/authentication/user_database_extra_fields/#authentication-user-database-extra-fields If you are returning that from userdb, you should not return it with `userdb_` prefix. Thank you, correct advice, my mistake. I have a combined password_query and made the mistake of adding userdb_ prefix in both. This has enabled the user limit for messages (as well as storage) and I can see over messages count causes defer like for over storage. Progress! Thank you for the help. For reference the quota warning for messages is also working (from dovecot.conf): quota_warning6 = messages=90%% quota-warning 90 %u Testing this was my objective which required setting a very low per user value for a test account. Thank you.
SQL user_query quota_rule for *:messages=
Hello, dovecot.conf sets the system default quota for storage and message count: plugin { ... quota_rule = *:storage=1G quota_rule2 = *:messages=1 ... } This quota_rule for the storage is overridden by an SQL user_query in dovecot-sql.conf: user_query = "SELECT ... '*:storage=' || maxstorage || 'M' AS userdb_quota_rule, ...;" Is it possible to override the message count limit with an SQL user_query (and password_query)? I tried returning userdb_quota_rule2 in the SQL but it did not work. "SELECT ... '*:messages=' || maxcount AS userdb_quota_rule2 ...;" Ref: https://doc.dovecot.org/configuration_manual/authentication/user_database_extra_fields/#authentication-user-database-extra-fields
Re: /usr/libexec/dovecot/anvil crashes immediately
On 08/02/2022 20:04, Friedrich Kink wrote: I built a dovecot package for openindiana (which is a Solaris derivative) from latest version 2.3.18. Everything compiles and builds fine without any issue. Even subsequent installation and startup of main dovecot process works as expected. But execution of /usr/libexec/dovecot/anvil immediately crashes. I can't see what is wrong from your trace. You might get more help on the OpenIndiana mailing list. I run dovecot on OmniOS (also a Solaris / illuminos derivative) without problem. I wonder why you are running dovecot on openindiana which is focused for desktop use; why not use OmniOS for a server? Your choice but you could run OpenIndiana global with a bhyve zone running OmniOS. James.
Re: ZFS storage and backup
On 15/11/2021 16:18, infoomatic wrote: Regarding storage I tend to use sdbox, from what I have read it seems to be the better option when using a COW filesystem compared to mdbox. One more https://doc.dovecot.org/admin_manual/mailbox_formats/ sdbox single-dbox, one message per file. mdbox multi-dbox, multiple messages per file. so I guess sdbox is better with ZFS. I could test each but I think I will find the IO used by dovecot is low for each. I have one user with 32,164 emails in INBOX and IO is not a problem. question is: compression at file system level or in dovecot storage? System. The OS compresses using all CPUs in a separate process. - does dovecot? Dovecot is smaller and simpler (--with-zlib=no etc). You can change the ZFS compression anytime. Text files remain plain text files even though they are compressed on disc. When available, zstd in ZFS should be a better option than gzip. The reason I am not sure to switch to ssds is that most servers are for non-profit organisations, sports clubs etc. - they also need some storage for pictures, their budget is quite low (so performance testing would only be done out of my interest), and if spinning rust with optimized settings suffices why not. As you have the HDDs already wait until there is a problem before fixing it. Over the internet I doubt anyone will notice and more importantly care enough to pay. Your HDDs might be old and about to fail so other factors rise in importance. Data security and continuity of service are more important than latency. Do you have enough RAM for read cache? A separate log for writes? L2ARC will only help if you have more active data than fits in RAM. James
Re: ZFS storage and backup
On 14/11/2021 14:14, infoomatic wrote: My setups are nothing special with few users, however, I would like to have a nice setup, maybe some of you could contribute to this thread. We are using slow spinning disks, but we may consider using ssds in a not-so-distant future. *) storages: any infos on ZFS options or whether to use mdbox or sdbox, and what configs/options regarding compression etc. OmniOS with ZFS here. I use maildir - just a personal choice and inertia, I have no performance data, no problem and no reason to change. I like being able to see emails as plain files. zfs set compress=gzip and no other changes from default, oh and atime=off on the whole machine. Email gzips well, most other ZFSes I leave on lz4. I say it is better to use the file system to compress rather than getting dovecot to do it. $ zfs get compress,compressratio,used ... NAME PROPERTY VALUE SOURCE .//vmail compressiongzip received .//vmail compressratio 1.82x - .//vmail used 8.55G - 25 mailbox users ("nothing special with few users"). I moved the storage from HDD (mirror plus log) to SSD (mirror) and no one noticed, not even me knowing it had been done and over a local network. I have enough RAM such that repeated reads are cached. I will use native ZFS encryption soon. I see no performance issues in test. Don't get hung up on ZFS tuning, mostly ZFS just works. *) backup: what is a best practice regarding backups? - using only the dovecot tools or leveraging the great features of ZFS (or both) with snapshots etc.? I use automated snapshots and zfs send/receive to a remote backup machine. I auto copy many ZFSes this way so it is minimal effort to do email too. James
Logging to remote server
My Google searches must be off, I couldn’t see how to do this. Is there a way to log to a local file, but also send to a remote server? Eg: Log_path = /var/log/dovecot.log, @mylogserver.example.com:514 Thanks, James
Re: Different userdb per inet_listener
On 28/07/2021 09:12, Miloslav Hůla wrote: Now we would like to disable authentication for Postfix (SMTP), but allow it for Dovecot (IMAP & POP3). Something like "receive-only". Is there any way we can configure different passdb for mentioned inet_listener? Or is there any variable with "auth requetor name" we can use in SQL query to differentiate the result? %s for service https://doc.dovecot.org/configuration_manual/config_file/config_variables/ Something like: password_query = "SELECT password, allow_nets, '*:storage=' || quota || 'M' AS userdb_quota_rule FROM mailbox WHERE username = '%n' AND domain = '%d' AND %Ls = true;" Note the "AND %Ls = true". The 'L' is for lower case. Add boolean columns for the services to your database.
Re: Different userdb per inet_listener
On 28/07/2021 09:12, Miloslav Hůla wrote: Now we would like to disable authentication for Postfix (SMTP), but allow it for Dovecot (IMAP & POP3). Something like "receive-only". Is there any way we can configure different passdb for mentioned inet_listener? Or is there any variable with "auth requetor name" we can use in SQL query to differentiate the result? %s for service https://doc.dovecot.org/configuration_manual/config_file/config_variables/ Something like: password_query = "SELECT password, allow_nets, '*:storage=' || quota || 'M' AS userdb_quota_rule FROM mailbox WHERE username = '%n' AND domain = '%d' AND %Ls = true;" Note the "AND %Ls = true". The 'L' is for lower case. Add boolean columns for the services to your database.
Re: [EXTERNAL] Sv: function for whitelisting IPs
On 15/07/2021 12:05, White, Daniel E. (GSFC-770.0)[NICS] wrote: The custom login script -- in Dovecot or Roundcube or … ? Is there any documentation for such scripting ? https://doc.dovecot.org/configuration_manual/authentication/auth_policy/ It uses an http interface so it is easy to implement with existing http toolkits. I wrote my own policy server in Java Jakarta EE9 because I can. You might prefer an existing policy server or write your own in your favourite http implementation language.
Re: libdict_lua linking issues
On 22/06/2021 12:30, Timo Sirainen wrote: libtool: link: gcc -std=gnu99 -m64 -march=x86-64 -fPIC -Os -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -m64 -o test-dict test-dict.o ./.libs/libdict.a ../lib-test/.libs/libtest.a ../lib/.libs/liblib.a -lsocket -lnsl -lsendfile gcc: error: ./.libs/libdict_lua.a: No such file or directory gmake[3]: *** [Makefile:630: test-dict-client] Error 1 Attached patch should work? You'll need to run autogen.sh again. Patching src/lib-dict/Makefile.in did the job. I don't know what is wrong with autoconf and automake - obviously I need a suite of tools to enable portability of autoconf, automake and libtool.
Re: libdict_lua linking issues
On 22/06/2021 12:30, Timo Sirainen wrote: And on OmniOS / Solaris it failed with: libtool: link: gcc -std=gnu99 -m64 -march=x86-64 -fPIC -Os -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -m64 -o test-dict test-dict.o ./.libs/libdict.a ../lib-test/.libs/libtest.a ../lib/.libs/liblib.a -lsocket -lnsl -lsendfile gcc: error: ./.libs/libdict_lua.a: No such file or directory gmake[3]: *** [Makefile:630: test-dict-client] Error 1 Attached patch should work? You'll need to run autogen.sh again. No, similar error, I am slowly investigating. I didn't run autogen.sh in the first place so I can't run it again. Running for the first time it moans about missing libtool: Warning: libtoolize does not appear to be available. This means that the automatic build preparation via autoreconf will probably not work. Preparing the build by running each step individually, however, should work and will be done automatically for you if autoreconf fails. ERROR: Unable to locate GNU Libtool. ERROR: To prepare the Dovecot build system from scratch, at least version 1.4.2 of GNU Libtool must be installed.
Re: [Dovecot-news] Dovecot v2.3.15 released
On 21/06/2021 17:39, Daniel J. Luke wrote: On Jun 21, 2021, at 7:20 AM, Timo Sirainen wrote: Here's a new release with some security fixes and quite a lot of other changes as well. * Removed support for Lua 5.2. Use version 5.1 or 5.3 instead. Looks like it doesn't want to build w/o lua now. On my MacOS system configure says: And on OmniOS / Solaris it failed with: libtool: link: gcc -std=gnu99 -m64 -march=x86-64 -fPIC -Os -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -m64 -o test-dict test-dict.o ./.libs/libdict.a ../lib-test/.libs/libtest.a ../lib/.libs/liblib.a -lsocket -lnsl -lsendfile gcc: error: ./.libs/libdict_lua.a: No such file or directory gmake[3]: *** [Makefile:630: test-dict-client] Error 1 I confirm v2.3.15 has an error. I tried combinations of --with-lua=no, --without-lua, etc, and also compiled lua and still it failed.
Re: NFS Locking Question
On 25/05/2021 16:46, Zane Zak wrote: We have a single exim server that delivers mail to mbox files in /var/mail. We have a separate server running dovecot that reads mail from these mbox files. ... Is there a configuration change that needs to be made on the dovecot I don't know about the NFS problem but change your configuration to use LMTP delivery. https://wiki.dovecot.org/LMTP/Exim Plus there are plenty of reasons to not use mbox.
Re: What imap ssl/auth settings work best with MS Outlook?
On 30/04/2021 08:13, @lbutlr wrote: When you enter your email address, it would be TRIVIAL to check the MX records for the domain and fill those in for the SMTP and IMAP servers, allowing users to more easily add (if needed) the domain prefix. Better to use DNS SVR records than guess from MX or domain. I provide email SVRs but does any mail client use them? https://tools.ietf.org/html/rfc6186 There is config-v1.1.xml, again I do not know which clients use, hence what I should provide, maybe I carry on providing as many methods as I can.
Re: Dovecot Replication Errors (only) when using tcps: as the mail_replica Protocol
On 18/11/2020 19:37, Aakash Patel wrote: Hello, I have two mail servers and am also experiencing sporadic replication errors over tcps, similar to Reuben. Each server is running Dovecot 2.3.11.3 (502c39af9) on Debian 10.6. *Log entries from MX1* Nov 18 00:39:26 mx1 dovecot: dsync-local(u...@example.com): Error: dsync(mx2.example.com): I/O has stalled, no activity for 600 seconds (last sent=mailbox, last recv=mailbox_state) Nov 18 00:39:26 mx1 dovecot: dsync-local(u...@example.com): Error: Timeout during state=sync_mails (send=mailbox recv=mailbox) Nov 18 06:39:32 mx1 dovecot: dsync-local(u...@example.com)<6bScGpwFtV+vEQAAPHKnuQ>: Error: dsync(mx2.example.com): I/O has stalled, no activity for 600 seconds (last sent=mailbox, last recv=mailbox_state) Nov 18 06:39:32 mx1 dovecot: dsync-local(u...@example.com)<6bScGpwFtV+vEQAAPHKnuQ>: Error: Timeout during state=sync_mails (send=mailbox recv=mailbox) *End* *Log entries from MX2* Nov 18 00:29:55 mx2 dovecot: dsync-local(u...@example.com): Error: Couldn't lock /var/vmail/u...@example.com/.dovecot-sync.lock: fcntl(/var/vmail/u...@example.com/.dovecot-sync.lock, write-lock, F_SETLKW) locking failed: Timed out after 30 seconds (WRITE lock held by pid 628) Nov 18 00:34:56 mx2 dovecot: dsync-local(u...@example.com)<9IKaB2KytF92AgAA5XpYKg>: Error: Couldn't lock /var/vmail/u...@example.com/.dovecot-sync.lock: fcntl(/var/vmail/u...@example.com/.dovecot-sync.lock, write-lock, F_SETLKW) locking failed: Timed out after 30 seconds (WRITE lock held by pid 628) Nov 18 00:39:26 mx2 dovecot: doveadm: Error: dsync(mx1.example.com): I/O has stalled, no activity for 600 seconds (last sent=mail_change (EOL), last recv=mailbox) Nov 18 06:39:32 mx2 dovecot: doveadm: Error: dsync(mx1.example.com): I/O has stalled, no activity for 600 seconds (last sent=mail_change (EOL), last recv=mailbox) *End* I have configured "replication_full_sync_interval = 1 hours", which explains why some of the sync errors occur at the same increment on the hour (if the error does occur). I've tested replication over tcps using either IPv6 or IPv4 -- this did not appear to make a difference. Changing replication to occur over tcp solves the issue (with "ssl = yes" commented out, as well). IMAP clients are primarily connecting to MX1 using SSL, which works well (SSL connections to MX2 also work). These are very low traffic machines at the moment (just 1 user as I continue testing). I've attached the output of "dovecot -n" from each server. Are there known bugs with replication using SSL? I'd appreciate any guidance. Thank you, AP For what it's worth, I had the same issue when setting this up a few weeks ago. I switched to using SSH based transport and it's been great ever since. Is that an option for you? dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u mail_replica = remote:r...@xx.xx.xx.xx Cheers James
Re: Odd replication behaviour
Solved. I knew this would happen. The act of writing it all out and including the configuration output gave me the solution. I am using lmtp to deliver mail from postfix to Dovecot. I was missing the notify and replication plugins from 20-lmtp.conf They were only present in 10-mail.conf as mail_plugins = notify replication Now, adding to 20-lmtp.conf: protocol lmtp { mail_plugins = sieve notify replication } Works fine now. Hope this helps someone else. Cheers James > On 31 Oct 2020, at 14:40, James Pattinson wrote: > > Hi, > > I have just built a new pair of similar machines both running CentOS 8.2 > (selinux disabled) and Dovecot 2.3.8 (9df20d2db). > > One machine is a VPS (host A) and one is on my home network (host B). The > idea is that they are set up in a master/master config with Dovecot > replication. > > I seem to have this 95% working but there is one strange issue I can’t work > out. > > Currently B is a perfect replica of A. I have pointed an instance of > Thunderbird at it, and I can see all my mails. If I delete any mails or > change any flags, I see the same changes almost instantly on the A side. > > PROBLEM: if host A receives a new mail, I don’t see it on B until I do > ‘something’ to change metadata, for example deleting any random email, or > marking an email as read on EITHER side causes the new email to appear almost > instantly on the B side. > > I would have expected emails on B to appear immediately. Am I doing something > wrong? > > Extra info - my mailboxes are in Maildir format with single OS user (vmail). > I have about 4000 emails in the Inbox and about 30k in other folders. > > There are only 5 users and I’m using passdb as the very simple backend. > > Replication is via doveadm on a specified port (not SSH). Some output from > dovecot -n is below. > > Cheers > James > > HOST A > > # 2.3.8 (9df20d2db): /etc/dovecot/dovecot.conf > # Pigeonhole version 0.5.8 (b7b03ba2) > # OS: Linux 4.18.0-193.28.1.el8_2.x86_64 x86_64 CentOS Linux release 8.2.2004 > (Core) xfs > # Hostname: hosta.domain > auth_mechanisms = plain login > doveadm_password = # hidden, use -P to show it > doveadm_port = 4040 > first_valid_uid = 1000 > mail_debug = yes > mail_home = /srv/vmail/%u > mail_location = maildir:/srv/vmail/%u > mail_plugins = notify replication > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope encoded-character > vacation subaddress comparator-i;ascii-numeric relational regex imap4flags > copy include variables body enotify environment mailbox date index ihave > duplicate mime foreverypart extracttext > mbox_write_locks = fcntl > namespace inbox { > inbox = yes > location = > mailbox Drafts { >auto = subscribe >special_use = \Drafts > } > mailbox Junk { >auto = subscribe >special_use = \Junk > } > mailbox Sent { >special_use = \Sent > } > mailbox "Sent Messages" { >special_use = \Sent > } > mailbox Trash { >auto = subscribe >special_use = \Trash > } > prefix = > } > passdb { > args = scheme=BLF-CRYPT username_format=%u /etc/dovecot/users > driver = passwd-file > } > plugin { > mail_replica = tcp:b.b.b.b:4040 > sieve = file:~/sieve;active=~/.dovecot.sieve > sieve_before = /var/mail/SpamToJunk.sieve > } > protocols = imap lmtp > service aggregator { > fifo_listener replication-notify-fifo { >group = root >mode = 0660 >user = vmail > } > unix_listener replication-notify { >group = root >mode = 0660 >user = vmail > } > } > service auth { > unix_listener /var/spool/postfix/private/auth { >group = postfix >mode = 0600 >user = postfix > } > } > service doveadm { > inet_listener { >port = 4040 > } > } > service lmtp { > unix_listener /var/spool/postfix/private/dovecot-lmtp { >group = postfix >mode = 0600 >user = postfix > } > } > service replicator { > process_min_avail = 1 > unix_listener replicator-doveadm { >mode = 0600 >user = vmail > } > } > ssl = required > ssl_cert = ssl_cipher_list = PROFILE=SYSTEM > ssl_dh = # hidden, use -P to show it > ssl_key = # hidden, use -P to show it > ssl_min_protocol = TLSv1.2 > ssl_prefer_server_ciphers = yes > userdb { > args = username_format=%u /etc/dovecot/users > default_fields = uid=vmail gid=mail home=/srv/vmail/%u > driver = passwd-file > } > protocol lmtp { > mail_plugins = sieve > } > protocol lda { > mail_plugins = notify replication sieve > } > > HOST B >
Odd replication behaviour
Hi, I have just built a new pair of similar machines both running CentOS 8.2 (selinux disabled) and Dovecot 2.3.8 (9df20d2db). One machine is a VPS (host A) and one is on my home network (host B). The idea is that they are set up in a master/master config with Dovecot replication. I seem to have this 95% working but there is one strange issue I can’t work out. Currently B is a perfect replica of A. I have pointed an instance of Thunderbird at it, and I can see all my mails. If I delete any mails or change any flags, I see the same changes almost instantly on the A side. PROBLEM: if host A receives a new mail, I don’t see it on B until I do ‘something’ to change metadata, for example deleting any random email, or marking an email as read on EITHER side causes the new email to appear almost instantly on the B side. I would have expected emails on B to appear immediately. Am I doing something wrong? Extra info - my mailboxes are in Maildir format with single OS user (vmail). I have about 4000 emails in the Inbox and about 30k in other folders. There are only 5 users and I’m using passdb as the very simple backend. Replication is via doveadm on a specified port (not SSH). Some output from dovecot -n is below. Cheers James HOST A # 2.3.8 (9df20d2db): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.8 (b7b03ba2) # OS: Linux 4.18.0-193.28.1.el8_2.x86_64 x86_64 CentOS Linux release 8.2.2004 (Core) xfs # Hostname: hosta.domain auth_mechanisms = plain login doveadm_password = # hidden, use -P to show it doveadm_port = 4040 first_valid_uid = 1000 mail_debug = yes mail_home = /srv/vmail/%u mail_location = maildir:/srv/vmail/%u mail_plugins = notify replication managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = } passdb { args = scheme=BLF-CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } plugin { mail_replica = tcp:b.b.b.b:4040 sieve = file:~/sieve;active=~/.dovecot.sieve sieve_before = /var/mail/SpamToJunk.sieve } protocols = imap lmtp service aggregator { fifo_listener replication-notify-fifo { group = root mode = 0660 user = vmail } unix_listener replication-notify { group = root mode = 0660 user = vmail } } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0600 user = postfix } } service doveadm { inet_listener { port = 4040 } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 user = vmail } } ssl = required ssl_cert =
Re: iterate_query with static userdb ?
On 17/05/2020 10:43, Laura Smith wrote: Because I wanted to avoid storing uid/gid/home in the database ? I use: user_query = "SELECT 'vmail' AS uid, 'vmail' AS gid, allow_nets, '*:storage=' || quota || 'M' AS quota_rule FROM mailbox WHERE username = '%n' AND domain = '%d';" ... uid and gid are not stored in my database but are returned by the query.
Re: dovecot sieve duplicates detection
On Wed, Dec 4, 2019, at 1:14 PM, Stephan Bosch via dovecot wrote: > > > On 17/08/2018 09:14, Stephan Bosch wrote: > > > > > > Op 14/05/2018 om 23:03 schreef James Cassell: > >> > >> On Mon, May 14, 2018, at 4:52 PM, Stephan Bosch wrote: > >>> > >>> Op 25/04/2018 om 22:49 schreef James Cassell: > >>>> On Wed, Apr 25, 2018, at 3:20 PM, Stephan Bosch wrote: > >>>>> Specify the ID used for duplicate checking explicitly using the > >>>>> :uniqueid argument (https://tools.ietf.org/html/rfc7352#section-3.1). > >>>>> Using the variables extenion, compose the uniqueid from the > >>>>> message-id > >>>>> and the mailbox name. > >>>>> > >>>> In my experience with dovecot's implementation, you can set the ID > >>>> only once in a script. If you try to filter duplicates based on > >>>> multiple IDs, only the first (or last, I don't remember) takes effect. > >>>> > >>> Do you have a detailed example of the supposed wrong behavior? > >>> > >> I don't have them readily available. Basically, the result of the > >> first duplicate test in a script is taken as the result of any future > >> duplicate test, even if the parameters to that future duplicate test > >> in the same script are different and would otherwise result in a > >> different output. The duplicate test is only evaluated once and its > >> results are substituted everywhere. > >> > >> For example, I might want to flag a message as a new conversation if > >> I have not seen another message with the same subject. In the same > >> script, I might want to discard messages that are exactly identical > >> including message ID among others. The dovecot behavior would be to > >> discard all messages that match a subject of previously received > >> message. > > > > I finally managed to review this issue and I can confirm that this is > > a bug. > > Fix released in 2.3.9. > Awesome! Thanks for the followup! V/r, James Cassell
Re: Duplicate e-mail with Dovecot and Sieve
On Tue, Nov 26, 2019, at 5:13 AM, Claudio Corvino via dovecot wrote: > > Hi everyone, > > we have a duplicate e-mail problem with Dovecot 2.2.13 (LMTP as MDA) on > a Debian 8 server. > > Our users are on external LDAP. > > We just need to setup a vacation responder (with Sieve) for our > employees with a redirect to an alias (a distribution list), let's call > it *l...@domain.com*, that comprehends even the user that just > activated the out of office. > > This generates a loop: when someone write to l...@domain.com every user > of the aforementioned distribution list receives a duplicate e-mail > generated from the sieve script of the user with vacation responder > active that contains the redirect to l...@domain.com. > > We just tried activating the duplicate extension into sieve script but > the redirected e-mail has a different MSG-ID and it's not recognized as > a duplicate by the server. > > Our dovecot conf is: > [snip] > An example of user sieve script is: > > *require ["vnd.dovecot.duplicate","copy","fileinto","vacation-seconds"]; > # rule:[Outofoffice] > if true > { > vacation :addresses "...@domain.tld" :subject "Automatic Reply" text: > * > > * > . > ; > fileinto "INBOX";* > > * redirect "l...@domain.com";* > > *}* > You've not used the duplicate test anywhere in your example. See the RFC for examples: https://tools.ietf.org/html/rfc7352#section-5 Specifically, example 3 might help: https://tools.ietf.org/html/rfc7352#section-5.3 but most simply, you can probably replace `if true` with `if not duplicate` V/r, James Cassell
Re: SQL iterate_query
On 25/10/2019 00:00, Daniel Miller via dovecot wrote: So, given that the complete address is used as the username I now use: iterate_query = SELECT username FROM mailbox https://doc.dovecot.org/configuration_manual/authentication/sql/#user-iteration ...says that iterate_query has two variants. You have complete addresses in a database column "username" so you need the first. The addresses must be return in a column named "user", hence add "AS user" to label your column. Try: iterate_query = SELECT username AS user FROM mailbox
Re: Error: SSL_accept() syscall failed
bit ECDH (P-384) OpenSSL 1.0.2e TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384, 384 bit ECDH (P-384) OpenSSL 1.1.0j (Debian) TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384, 384 bit ECDH (P-384) OpenSSL 1.1.1b (Debian) TLSv1.3 TLS_AES_256_GCM_SHA384, 384 bit ECDH (P-384) Thunderbird (60.6) TLSv1.3 TLS_AES_128_GCM_SHA256, 384 bit ECDH (P-384) Done 2019-10-11 07:31:08 [ 170s] -->> 3.222.54.62:993 (kumo.kites.org) <<-- On 10/11/19 7:22 AM, C. James Ervin via dovecot wrote: In setting up my new mail server, I am getting the following in the logs: Oct 11 07:10:59 kumo dovecot[5704]: imap-login: Disconnected (no auth attempts in 0 secs): user=<>, rip=24.53.79.10, lip=172.26.12.90, *TLS handshaking: SSL_accept() syscall failed: Success*, session=
Error: SSL_accept() syscall failed
In setting up my new mail server, I am getting the following in the logs: Oct 11 07:10:59 kumo dovecot[5704]: imap-login: Disconnected (no auth attempts in 0 secs): user=<>, rip=24.53.79.10, lip=172.26.12.90, *TLS handshaking: SSL_accept() syscall failed: Success*, session= I have tried various ssl_protocols entries, but for now have defaulted back to ssl_protocols = !SSLv3 (the "out of the box" setting). The certificate (ssl_cert = valid and the same one used by postfix. However, I belive the error appeared only after removing the self signed certificate and installing the one I purchased. Warning: though I was a sysadmin a long long time, it has been a while! I have not run a mail server in 15 years, but am moving my mail back "in-house" as I approach retirement! I searched for a forum where I could post this issue, but only found this list. # 2.2.33.2 (d6601f4ec): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.21 (92477967) # OS: Linux 4.15.0-1051-aws x86_64 Ubuntu 18.04.1 LTS auth_mechanisms = plain login mail_location = maildir:~/Maildir mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = " imap sieve pop3" service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } } ssl_cert = was automatically rejected:%n%r } protocol imap { imap_client_workarounds = delay-newmail mail_max_userip_connections = 10 } protocol pop3 { mail_max_userip_connections = 10 pop3_client_workarounds = outlook-no-nuls oe-ns-eoh }
Re: Case sensitivity in :addresses in sieve vacation scripts
On Tue, Oct 8, 2019, at 7:58 AM, Julian Kippels via dovecot wrote: > Hi, > > I have recently updated from Dovecot 2.2 to 2.3. Since I have noticed > that vacation responses from sieve are not working the same anymore. > For example, my sieve script looks like this: > > vacation :days 1 :addresses ["kipp...@hhu.de","julian.kipp...@hhu.de"] > > it used to be that I got a vacation response if I sent a mail to > kipp...@hhu.de and kipp...@hhu.de. Now I only get a response for > kipp...@hhu.de, not for kipp...@hhu.de. > I cant say for sure, but I suppose this behaviour changed with the > update. If not, why could this have happened? And in any case, how can > I enable case insensitivity? > I took a look at the RFCs. It appears that this change is not in accordance with the relevant standards. SIEVE says 'the "i;ascii-casemap" comparator (which treats uppercase and lowercase characters in the US-ASCII subset of UTF-8 as the same). If left unspecified, the default is "i;ascii-casemap".' https://tools.ietf.org/html/rfc5228#section-2.7.3 Since there is no mention of comparators in the Vacation RFC, it should fallback to case-insensitive: https://tools.ietf.org/html/rfc5230 You could try working around the issue by adding to your `vacation` statement: `:comparator "i;ascii-casemap"` -- but in any case, I'd file a bug about the non-standard behavior. V/r, James Cassell > Thanks > Julian > >
Off-site cloud backup (eg Amazon S3, Wasabi)
I’d like to synchronise our mail store to an Amazon S3 bucket or similar, for protection from ransomware. Server is running on macOS with maildir. Should I use a Dovecot process, rsync, a Mac app like Arq or Jungle Disk, or something else? Anyone have experience or opinions on this? Thanks, James. smime.p7s Description: S/MIME cryptographic signature
Solr, Dovecot & macOS / iOS
I’m thinking of getting Solr working with my Dovecot server. Server is new 6-core Mac Mini, mail store of over 1/2 TB. Mailboxes with 100s of thousands of messages. But I’m not sure if: a) it will make enough of a difference and b) does Mail.app and other mail clients on Macs or iOS devices perform searches on their local copy of mail or does it just send a search request to the server? I’m guessing the searches are done locally so no point in Solr? Does anyone know? Thanks, James. smime.p7s Description: S/MIME cryptographic signature
Re: auth-policy crashing
On 07/08/2019 11:19, James via dovecot wrote: My more simplistic policy does not need both. I perform whitelist, blacklist, geo and greylist ...and DNSBL which where I started with the policyserver, "Can dovecot do DNSBL?", only indirectly via a policyserver. This is better as most pass white list or fail geo local checks before doing the external DNS lookup.
Re: auth-policy crashing
On 07/08/2019 11:02, Aki Tuomi via dovecot wrote: before and after auth? roundcube webmail reports an error with only auth_policy_check_before_auth. I cannot see why. The simple and lazy solution is to use double auth_policy_check_! ... The double-check is for places which want to implement something like COS or want to perform validations in policy server *after* we know the user identity. The first check is done before we even know if the user or the credential(s) are valid. I can see why both before and after are options. My more simplistic policy does not need both. I perform whitelist, blacklist, geo and greylist and do not cross reference these with the user. I can't see why roundcubemail fails without both. The IMAP exchange with roundcubemail should not be aware of the policy server. I was spending [wasting] too much time on looking for an answer and gave up.
Re: auth-policy crashing
On 06/08/2019 06:46, Aki Tuomi via dovecot wrote: On 2.8.2019 13.45, James via dovecot wrote: My auth process is dumping core. This happens several times per day ... There is an easy fix for this, attached. Patch applied; no core dump in 24 hours. This appears to have fixed the problem. I found that it crashed when the policy server responded too quickly. As the before and after auth command=allow request are the same I cache the first, leading to a fast second response. Removing the cache (nginx proxy_cache ...) must change the timings and circumvented the crash. Why use both check before and after auth? roundcube webmail reports an error with only auth_policy_check_before_auth. I cannot see why. The simple and lazy solution is to use double auth_policy_check_! Thank you Aki for looking at this and finding a solution so quickly.
Re: auth-policy crashing
On 02/08/2019 11:45, James via dovecot wrote: My auth process is dumping core. This happens several times per day but dovecot can operate normally for hours between errors. The crash occurs in src/auth/auth-policy.c, line 356: t@1 (l@1) program terminated by signal SEGV (no mapping at the fault address) Current function is auth_policy_parse_response 356 context->request->policy_refusal = FALSE; Further tracking shows this sets context->request to NULL: "src/lib/iostream.c" line 54 array_foreach(>destroy_callbacks, dc) dc->callback(dc->context); Very occasionally I see: Aug 3 11:00:35 mailhost dovecot: [ID 702911 mail.crit] auth: Panic: file http-client-request.c: line 283 (http_client_request_unref): assertion failed: (req->refcount > 0) Swapping keep-alive on/off changes crashing from very approximately once per day to some per hour. I guess there is some fundamental thread clash or keep alive time out clean-up failure. James.
auth-policy crashing
My auth process is dumping core. This happens several times per day but dovecot can operate normally for hours between errors. The crash occurs in src/auth/auth-policy.c, line 356: t@1 (l@1) program terminated by signal SEGV (no mapping at the fault address) Current function is auth_policy_parse_response 356 context->request->policy_refusal = FALSE; ...context->request is null. Add markers to the code... } if (context->request == NULL) fprintf(stderr, " context->request == NULL\n"); i_stream_unref(>payload); if (context->request == NULL) fprintf(stderr, " context->request == NULL\n"); if (context->parse_error) { context->result = (context->set->policy_reject_on_fail ? -1 : 0); } if (context->request == NULL) fprintf(stderr, " context->request == NULL\n"); context->request->policy_refusal = FALSE; if (context->result < 0) { ...gives at the time of the crash... Aug 1 14:25:44 mailhost dovecot: [ID 702911 mail.error] auth: Error: context->request == NULL Aug 1 14:25:44 mailhost dovecot: [ID 702911 mail.error] auth: Error: context->request == NULL ...so context->result is not null before the call (no 222) to i_stream_unref but is after. dovecot.conf has: auth_policy_server_url = http://policyserver.lan/ auth_policy_server_timeout_msecs = 3000 auth_policy_hash_nonce = Ohr9phaeSeip2Pahaez2raiGohxoo5Ia auth_policy_request_attributes = remote=%{rip} auth_policy_check_before_auth = yes auth_policy_check_after_auth = yes auth_policy_report_after_auth = yes To simplify the problem I used a dummy policy server, in nginx.conf: location / { default_type application/json; return 200 "{\"status\":0,\"msg\":\"accepted\"}"; } however no matter what rubbish a policy server sends back it should not cause dovecot to crash. I've tried 32 and 64 bit and two compilers (gcc and SunStudio), all result in crashes. Adding: keepalive_timeout 0; ...to nginx.conf appears to reduce the crashes. It happens with a variety of users and with debug output I see no pattern. James.
Re: Dovecot 2.3.6 on Solaris10: build issues, segfaults
On 09/07/2019 06:35, Aki Tuomi via dovecot wrote: As for 1st, isn't gcc 3 rather old? As is Solaris 10. Solaris 10 predates gcc4 and comes/came with gcc3. No one says we have keep using the system gcc3 exclusively. 9.1.0 works too. Solaris 10 03/05 = March 2005, release January 31, 2005 [1] gcc 4.0.0: April 20, 2005 [2] $ pkginfo -l SUNWgcc PKGINST: SUNWgcc NAME: gcc - The GNU C compiler CATEGORY: system ARCH: i386 VERSION: 11.10.0,REV=2005.01.08.01.09 BASEDIR: / VENDOR: Oracle Corporation DESC: GNU C - The GNU C compiler 3.4.3 PSTAMP: sfw10-patch-x20121120060015 ... 1. https://en.wikipedia.org/wiki/Solaris_%28operating_system%29#Version_history "supported until 2021" 2. https://www.gnu.org/software/gcc/releases.html
Re: Dovecot 2.3.6 on Solaris10: build issues, segfaults
On 09/07/2019 01:02, Joseph Tam via dovecot wrote: Issue 2) Cannot build with --enable-hardening Using gcc 9.1.0, "configure" step fails because fd passing was broken, but the real problem was a compilation failure when "--enable-hardening" is used. Demonstration: See: https://dovecot.org/pipermail/dovecot/2019-January/114121.html Issue 3) dovecot/doveconf segfaults on startup It crashes here while processing dovecot.conf, as does "doveconf" (settings-parser.c:1519 in setting_copy()) *dest_size = *src_size; It appears *src_size is not an 8-byte address aligned (0x5597c). It inherits this value from the calling routine as the sum of "set" (8-byte aligned) + "def->offset"=20 => misaligned address. 32 bit or 64bit? cflags? I use 32 bit cc 12.6 and have no problem. My test with gcc 9.1.0 didn't dump core either.
Re: Compile problem on Solaris
On 10/06/2019 14:40, Roderick Johnstone via dovecot wrote: Compiling dovecot 2.2.36.1 with the native compiler on Solaris 10 is giving the error below. ... -m64 -c -o sha3.lo sha3.c libtool: compile: cc -DHAVE_CONFIG_H -I. -I../.. -I/export/home/exim_build_20190605/build_local/include -m64 -c sha3.c -KPIC -DPIC -o .libs/sha3.o "sha3.h", line 49: warning: unnamed struct member ... Untested: try naming the union. My own builds (dovecot 2.3.6) get through this. What compiler version? 12.6 is current and does not issue the unnamed struct warning, eg, 12.3 does. struct sha3_ctx { uint64_t saved; /* the portion of the input message that we * didn't consume yet */ union { /* Keccak's state */ uint64_t s[SHA3_KECCAK_SPONGE_WORDS]; uint8_t sb[SHA3_KECCAK_SPONGE_WORDS * 8]; } X;
Re: v2.3.6 Configure fail 'libssl not found' macOS
> On 1 May 2019, at 4:38 pm, James Brown via dovecot <mailto:dovecot@dovecot.org>> wrote: > > No problems with 2.3.5, but when I try to compile 2.3.6 with: > > ./configure --with-mysql --with-ssl=openssl > > I get: > > checking for dlopen... yes > checking for SSL_read in -lssl... no > configure: error: Can't build with OpenSSL: libssl not found > > $ locate libssl > /Applications/BBEdit.app/Contents/XPCServices/GitIgnoreXPCService.xpc/Contents/Frameworks/libssl.dylib > /Applications/Navicat for MySQL.app/Contents/Frameworks/libssl.1.0.0.dylib > /Applications/assp/assp.mod/bin/libssl32.dll > /usr/lib/libssl.0.9.7.dylib > /usr/lib/libssl.0.9.8.dylib > /usr/lib/libssl.35.dylib > /usr/lib/libssl.43.dylib > /usr/lib/libssl.44.dylib > /usr/lib/libssl.dylib > /usr/local/Cellar/openssl/1.0.2r/lib/libssl.1.0.0.dylib > /usr/local/Cellar/openssl/1.0.2r/lib/libssl.a > /usr/local/Cellar/openssl/1.0.2r/lib/libssl.dylib > /usr/local/Cellar/openssl/1.0.2r/lib/pkgconfig/libssl.pc > /usr/local/Cellar/openssl@1.1/1.1.1b/lib/libssl.1.1.dylib > /usr/local/Cellar/openssl@1.1/1.1.1b/lib/libssl.a > /usr/local/Cellar/openssl@1.1/1.1.1b/lib/libssl.dylib > /usr/local/Cellar/openssl@1.1/1.1.1b/lib/pkgconfig/libssl.pc > /usr/local/lib/dovecot/libssl_iostream_openssl.a > /usr/local/lib/dovecot/libssl_iostream_openssl.la > <http://libssl_iostream_openssl.la/> > /usr/local/lib/dovecot/libssl_iostream_openssl.so > > Any suggestions? > > Thanks, > > James. Sorry for the noise. export CPPFLAGS="-I/usr/local/opt/mysql@5.7/include/mysql -I/usr/local/Cellar/openssl@1.1/1.1.1b/include" export LDFLAGS="-L/usr/local/opt/mysql@5.7/lib -L/usr/local/opt/openssl@1.1/lib" ./configure --with-mysql --with-ssl=openssl Worked. James. smime.p7s Description: S/MIME cryptographic signature
v2.3.6 Configure fail 'libssl not found' macOS
No problems with 2.3.5, but when I try to compile 2.3.6 with: ./configure --with-mysql --with-ssl=openssl I get: checking for dlopen... yes checking for SSL_read in -lssl... no configure: error: Can't build with OpenSSL: libssl not found $ locate libssl /Applications/BBEdit.app/Contents/XPCServices/GitIgnoreXPCService.xpc/Contents/Frameworks/libssl.dylib /Applications/Navicat for MySQL.app/Contents/Frameworks/libssl.1.0.0.dylib /Applications/assp/assp.mod/bin/libssl32.dll /usr/lib/libssl.0.9.7.dylib /usr/lib/libssl.0.9.8.dylib /usr/lib/libssl.35.dylib /usr/lib/libssl.43.dylib /usr/lib/libssl.44.dylib /usr/lib/libssl.dylib /usr/local/Cellar/openssl/1.0.2r/lib/libssl.1.0.0.dylib /usr/local/Cellar/openssl/1.0.2r/lib/libssl.a /usr/local/Cellar/openssl/1.0.2r/lib/libssl.dylib /usr/local/Cellar/openssl/1.0.2r/lib/pkgconfig/libssl.pc /usr/local/Cellar/openssl@1.1/1.1.1b/lib/libssl.1.1.dylib /usr/local/Cellar/openssl@1.1/1.1.1b/lib/libssl.a /usr/local/Cellar/openssl@1.1/1.1.1b/lib/libssl.dylib /usr/local/Cellar/openssl@1.1/1.1.1b/lib/pkgconfig/libssl.pc /usr/local/lib/dovecot/libssl_iostream_openssl.a /usr/local/lib/dovecot/libssl_iostream_openssl.la <http://libssl_iostream_openssl.la/> /usr/local/lib/dovecot/libssl_iostream_openssl.so Any suggestions? Thanks, James. smime.p7s Description: S/MIME cryptographic signature
Re: Dovecot release v2.3.6
On 30/04/2019 14:21, Aki Tuomi via dovecot wrote: https://dovecot.org/releases/2.3/dovecot-2.3.6.tar.gz Trivial but... "mail-index-transaction-update.c", line 198: void function cannot return value Thanks. --- ../original/src/lib-index/mail-index-transaction-update.c 2019-04-30 13:25:06.0 + +++ src/lib-index/mail-index-transaction-update.c 2019-04-30 14:49:09.517684762 + @@ -195,7 +195,8 @@ uint32_t first_uid, ARRAY_TYPE(seq_range) *uids_r) { - return mail_index_append_finish_uids_full(t, first_uid, first_uid, uids_r); + mail_index_append_finish_uids_full(t, first_uid, first_uid, uids_r); + return; } void mail_index_append_finish_uids_full(struct mail_index_transaction *t,
Re: Mail account brute force / harassment
On 12/04/2019 08:42, Aki Tuomi via dovecot wrote: On 12.4.2019 10.34, James via dovecot wrote: On 12/04/2019 08:24, Aki Tuomi via dovecot wrote: Weakforced uses Lua so you can easily integrate DNSBL support into it. How does this help Dovecot block? A link to some documentation or example perhaps? https://wiki.dovecot.org/Authentication/Policy You can configure weakforced to return status -1 when DNSBL matches, which causes the user authentication to fail before any other processing happens. Thank you. I will study this - although I dispute your "easily"! James.
Re: Mail account brute force / harassment
On 12/04/2019 08:24, Aki Tuomi via dovecot wrote: Weakforced uses Lua so you can easily integrate DNSBL support into it. How does this help Dovecot block? A link to some documentation or example perhaps? We will not add DNSBL support to dovecot at this time. Is there a reason why you will not support this RFE?
Re: Mail account brute force / harassment
On 11/04/2019 14:33, Anton Dollmaier via dovecot wrote: Which is why a dnsbl for dovecot is a good idea. I do not believe the agents behind these login attempts are only targeting me, hence the addresses should be shared via a dnsbl. Probably there's an existing solution for both problems (subsequent attempts and dnsbl): https://github.com/PowerDNS/weakforced "The goal of 'wforce' is to detect brute forcing of passwords across many servers" The problem is not detecting but blocking. Dovecot has no mechanism for using the data; Dovecot needs DNSBL capability. I tested a small sample of my IMAP hackers using the lists I use for SMTP blocking [1] and enough are in these list to make them worth using. Extra detection is not needed as many of these addresses are already known - maybe even by using weakforced. James. 1. exim dnsblist: https://www.exim.org/howto/rbl.html https://www.exim.org/exim-html-current/doc/html/spec_html/ch-access_control_lists.html
Re: Mail account brute force / harassment
On 11/04/2019 12:49, Marc Roos via dovecot wrote: Yes indeed, we have already own dnsbl's for smtp and ssh/ftp access. How do you have one setup for dovecot connections? Two answers: 1. I wrote my own very simple implementation but it does not share other people's data. Sharing the key to viability so it is/was a pointless exercise. Without sharing a hacker gets at least one free shot per server per address. With sharing it is closer to one per address and less with honeypots. 2. I said "dnsbl for dovecot is a good idea", an idea. When this was raised previously we were told it was not needed and it can all be done with tcp wrappers, fail2ban and allow_nets. https://dovecot.org/list/dovecot/2013-July/091236.html https://dovecot.org/list/dovecot/2014-June/096662.html
Re: Mail account brute force / harassment
On 11/04/2019 11:43, Marc Roos via dovecot wrote: A. With the fail2ban solution - you 'solve' that the current ip is not able to access you It is only a solution if there are subsequent attempts from the same address. I currently have several thousand addresses blocked due to dovecot login failures. My firewall is set to log these so I can see that few repeat, those that do repeat have intervals of >1 week. Blocking these has minimal effect (other than to clog fail12ban and the firewall). - it will continue bothering other servers and admins Which is why a dnsbl for dovecot is a good idea. I do not believe the agents behind these login attempts are only targeting me, hence the addresses should be shared via a dnsbl.
Re: sql table definitions
On 02/04/2019 05:42, Richard Hector via dovecot wrote: I'm using PostgreSQL for my auth db. I used the example CREATE TABLE statement in the config file, but now I find the fields are too short. I assume dovecot will be fine with 'text' type columns replacing the varchars? Or failing that, I can change the length of the varchar fields? ALTER TABLE $table ALTER COLUMN $column TYPE VARCHAR($newlenth); My tables are very different from the dovecot suggested tables and it works, just make sure the queries in dovecot-sql.conf correspond. My tables are different because they hold additional information for routing (the db is shared with exim).
dovecot.conf "local hostname" uses only one resolved value
https://wiki.dovecot.org/SSL/DovecotConfiguration#Different_certificates_per_IP_and_protocol says: local 192.0.2.10 { # instead of IP you can also use hostname, which will be resolved However if the name resolves to multiple values only one is used. Test. Choose any name with multiple values, I created a local name: $ nslookup multi.lan Server: 127.0.0.1 Address:127.0.0.1#53 Name: multi.lan Address: 192.168.1.2 Name: multi.lan Address: 192.168.1.3 Name: multi.lan Address: 192.168.1.1 Minimal dovecot.conf: local multi.lan { protocol imap { ssl_cert = In my real case with A and records, only the record is used. Testing with "openssl s_client -starttls imap ..." gives me the wrong certificate for the IPv4 address. Workaround: specify all addresses and do not use lookup. James.
“Message cannot be displayed because of the way it is formatted” iOS
Have updated our mail server and are now running Dovecot 2.3.4. Things are mostly working OK, but occasionally I have users on latest iOS getting this message: “this message cannot be displayed because of the way it is formatted. Ask the sender to send it again using a different format or email program. multipart/alternative” The email seems to open OK on the macOS version of Mail.app. Also getting “Loading...” displayed in body section of the app with content never displayed. Most emails are fine, but just get this strange ones occasionally. Our previous mail server was much older, with older version of Dovecot, etc. This server never used to have these issues. Just wondering if anyone has any ideas or suggestions? Can’t see anything in the logs. Thanks, James. smime.p7s Description: S/MIME cryptographic signature
same account imap and pop3
I have an imap and pop3 for the same account. When I delete a message from imap and then fetch the mail from the pop3 account, it retrieves the deleted message. I am using thunderbird for the MTA. Does anyone know why it happens? It could be a timing issue.
Re: "unknown user - trying the next userdb" Info in log
On 30 Jan 2019, at 4:35 pm, Aki Tuomi mailto:aki.tu...@open-xchange.com>> wrote: > >> >> On 30 January 2019 at 07:12 James Brown < jlbr...@bordo.com.au >> <mailto:jlbr...@bordo.com.au>> wrote: >> >> >> >> My settings: >> ... >> >> userdb { >> >> driver = passwd >> >> } >> >> userdb { >> >> driver = prefetch >> >> } >> >> userdb { >> >> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext >> >> driver = sql >> >> } >>> Well... there is that usetdb passwd which seems bit extraneous. >>> --- >>> Aki Tuomi >> > I'd remove the > > userdb { > driver = passwd > } > > section > --- > Aki Tuomi Thanks Aki - the trick was finding where that setting was! Found it in auth-system.conf.ext. Commented it out and all works perfectly now. Thanks again Aki, James.
Re: "unknown user - trying the next userdb" Info in log
On 30 Jan 2019, at 10:57 am, Stephan Bosch mailto:step...@rename-it.nl>> wrote: > > Op 30/01/2019 om 00:06 schreef James Brown via dovecot: >> >>> On 30 Jan 2019, at 9:24 am, Stephan Bosch >> <mailto:step...@rename-it.nl>> wrote: >>> >>> >>> >>> Op 29/01/2019 om 01:51 schreef James Brown via dovecot: >>>> I’ve set up a new mail server. Auth-worker is giving me this error all the >>>> time: >>>> >>>> auth-worker(11160): Info: passwd(sa...@bordo.com.au >>>> <mailto:sa...@bordo.com.au> <mailto:sa...@bordo.com.au >>>> <mailto:sa...@bordo.com.au>>,10.0.0.54,): unknown user >>>> - trying the next userdb >>>> >>> That is not an error. As the "Info: " prefix indicates, this is an >>> innocuous informational message. >>> >>> Regards, >>> >>> Stephan. >>> >> Thanks Stephan. >> >> I know it’s not a major thing, it’s just that it is every third line of my >> log. >> >> Ie I get the auth-worker unknown user, then successful imap-login, then the >> imap logout. >> >> So I’m just trying to figure out what is causing the auth-worker lookup >> failure so I can fix it. I did not have this in my old mail server, just the >> new one I’ve setup. >> >> Is it a database issue or a Dovecot config issue? If so where? > > Ýou can enable auth_verbose and auth_debug to get an idea what Dovecot is > doing. You should post your `dovecot -n` to get more help on this issue. > > Regards, > > Stephan. Will look at auth_debug. It is a lot to go through! My settings: $ sudo dovecot -n Password: # 2.3.4 (0ecbaf23d): /usr/local/etc/dovecot/dovecot.conf # OS: Darwin 18.2.0 x86_64 apfs # Hostname: mail.bordo.com.au <http://mail.bordo.com.au/> auth_debug = yes auth_verbose = yes auth_verbose_passwords = plain default_internal_user = jlbrown default_login_user = jlbrown first_valid_gid = 0 first_valid_uid = 102 last_valid_gid = 501 last_valid_uid = 105 log_path = /var/log/dovecot.log mail_gid = 102 mail_location = maildir:/usr/local/virtual/%d/%u mail_uid = 102 namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } protocols = imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = _postfix mode = 0660 user = _postfix } unix_listener auth-userdb { mode = 0660 user = _vmail } } service imap-login { process_limit = 512 } service stats { fifo_listener stats-mail { mode = 0666 } fifo_listener stats-user { mode = 0666 } unix_listener stats-writer { mode = 0666 } } ssl_cert =
Re: "unknown user - trying the next userdb" Info in log
> On 30 Jan 2019, at 9:24 am, Stephan Bosch wrote: > > > > Op 29/01/2019 om 01:51 schreef James Brown via dovecot: >> I’ve set up a new mail server. Auth-worker is giving me this error all the >> time: >> >> auth-worker(11160): Info: passwd(sa...@bordo.com.au >> <mailto:sa...@bordo.com.au>,10.0.0.54,): unknown user - >> trying the next userdb >> > > That is not an error. As the "Info: " prefix indicates, this is an innocuous > informational message. > > Regards, > > Stephan. > Thanks Stephan. I know it’s not a major thing, it’s just that it is every third line of my log. Ie I get the auth-worker unknown user, then successful imap-login, then the imap logout. So I’m just trying to figure out what is causing the auth-worker lookup failure so I can fix it. I did not have this in my old mail server, just the new one I’ve setup. Is it a database issue or a Dovecot config issue? If so where? Thanks, James.
"unknown user - trying the next userdb" Info in log
I’ve set up a new mail server. Auth-worker is giving me this error all the time: auth-worker(11160): Info: passwd(sa...@bordo.com.au <mailto:sa...@bordo.com.au>,10.0.0.54,): unknown user - trying the next userdb Next line is from imap-login with successful login. Using MySQL as user database. Auth-sql.comf.ext is: passdb { driver = sql # Path for SQL configuration file, see example-config/dovecot-sql.conf.ext args = /usr/local/etc/dovecot/dovecot-sql.conf.ext } # "prefetch" user database means that the passdb already provided the # needed information and there's no need to do a separate userdb lookup. # userdb { driver = prefetch } userdb { driver = sql args = /usr/local/etc/dovecot/dovecot-sql.conf.ext } # If you don't have any user-specific settings, you can avoid the user_query # by using userdb static instead of userdb sql, for example: # #userdb { #driver = static #args = uid=vmail gid=vmail home=/var/vmail/%u #} Any ideas why it is gives me this error before logging in? Where should I look? Thanks, James.
Re: Sieve "OOO" configuration
Look at the currentdate test https://tools.ietf.org/html/rfc5260#section-5 V/r, James Cassell On Sat, Jan 5, 2019, at 11:26 AM, Jerry wrote: > I am able to get sieve issuing an "out of office"message correctly. > However, I want to configure it to send an "OOO" message only during > certain dates, say on weekends, or only between certain hours. I can do > that manually; however, was wondering if there is any automatic method > available that could handle this chore. > > Thanks! > > -- > Jerry
Re: Compiling Dovecot on Solaris 11 fails
On 03/01/2019 20:42, Sami Ketola wrote: does work just fine on my solaris 11 x86 box. ... my test system is solaris 11.3 and I have gcc 4.5.2. Old versions of gcc do not have -mfunction-return=thunk 7 and 8 have it.
Re: Compiling Dovecot on Solaris 11 fails
On 03/01/2019 16:45, Andrew Watkins wrote: Tried to build dovecot-2.3.4 on Solaris 11 x86 and it fails at configure part. I just went checked and it last version it works on is v2.3.2.1 ... checking Linux compatible mremap()... no checking whether shared mmaps get updated by write()s... no checking whether fd passing works... no configure: error: fd passing is required for Dovecot to work In think it's thunk (but I forget exactly what aspect). Try: ./configure --disable-hardening ... Or build with cc. (Sorry, it's too late for me to do another test today.)
Re: Several problems on Solaris10
On 31/12/2018 17:28, Pierluigi Frullani wrote: > My version is 2.2.13 ( it was the last one, at the time of the first > server setup ). 2.2.13 is from around May 2014. It worked but I can't see why you wouldn't switch to the latest 2.3.4. (You might be seeing what I can't and your question hasn't explained.) That's the date for installation. I was using the courier-imap and switched to dovecot. Not changed since then. So use the latest version. > I have seen that ( it seems ) the new solaris don't honour the > LD_LIBRARY_PATH. I'm sure it does but you shouldn't need it anyway. Believe me, it doesn't :( Test: $ cat > sub.c << EOF void sub() {} EOF $ cat > main.c << EOF void sub(); int main() { sub(); } EOF $ cc -G -o libsub.so sub.c -- link with no run path in the executable, fails with no other help $ cc main.c -L. -lsub $ ./a.out ld.so.1: a.out: fatal: libsub.so: open failed: No such file or directory zsh: killed ./a.out -- succeeds with LD_LIBRARY_PATH set $ LD_LIBRARY_PATH=. ./a.out -- link with a run path, succeeds on its own $ cc main.c -L. -R. -lsub $ ./a.out $ dump -Lv a.out | grep RUNPATH [5] RUNPATH . -- to see the logic when finding a library use "ldd -s ..." $ LD_LIBRARY_PATH=. ldd -s ./a.out find object=libsub.so; required by a.out search path=. (LD_LIBRARY_PATH) trying path=./libsub.so libsub.so => ./libsub.so -- clean up: $ rm sub.c main.c libsub.so a.out The problem ( as usual ) arise with the openssl libs that solaris ships, Build your own openssl. 1. Do not put your files in /use/local. You will clash with someone else thinking it is the place to put personal stuff. man filesystem: "/opt Root of a subtree for add-on application packages." Being that I'm the only one administering this machine I'm sure it would not ;) You will appreciate the separation of system and add-ons. Solaris 10 sparse zones can make /usr read only. It really is the "right" way. Would you ( or could ) show me where to replace -R and -L on Makefile(s) to fix the runpath trouble ? I've tried the following: CPPFLAGS="-I/usr/local/clucene/include -I/usr/local/libtextcat/include -I/usr/local/openssl-1.0.1e/include" LDFLAGS="-L/usr/local/clucene/lib -L/usr/local/libtextcat/lib -L/usr/local/openssl-1.0.1e/lib -R/usr/local/openssl-1.0.1e/lib " ./configure --prefix=/usr/local/dovecot --with-ssl=yes --with-stemmer --with-lucene --with-zlib and it seems to work: root@puma dump -Lv ./src/imap-login/.libs/imap-login | grep RUN [14]RUNPATH /usr/local/dovecot/lib/dovecot:/usr/local/lib:/usr/local/openssl-1.0.1e/lib:/usr/ccs/lib:/lib:/usr/lib:/usr/sfw/lib It would be easier if you put openssl and your other software under a common prefix, $PREFIX/lib, (not a subdirectory). The SONAME should sort out versions. If you are doing a simple build you will get away with the standard handling of -L and -R. $ PREFIX=/opt/PREFIX $ export LDFLAGS="-R/opt/FIRST/lib:${PREFIX}/lib" $ ./configure --prefix=${PREFIX} ... Gives RPATH: /opt/PREFIX/lib/dovecot:/opt/FIRST/lib:/opt/PREFIX/lib which I think will work for you. Problems with libtool are when we require: + /opt/PREFIX/lib/$ISALIST at the front before /opt/PREFIX/lib/ + a non standard lib is needed before the normal one instead of libtool using /opt/PREFIX/lib as the first In general it is easier to let libtool think it is helping but ignore it. Create a wrapper script around cc to rewrite the args back to what they were before configure+libtool rearranged, in this case drop the -L and -R given and substitute the values in $LDFLAGS. More reading: https://blogs.oracle.com/solaris/ldlibrarypath-just-say-no-v2 https://docs.oracle.com/cd/E86824_01/html/E54763/ld.so.1-1.html https://docs.oracle.com/cd/E19683-01/816-1386/chapter3-13312/ Avoid crle, you don't need it and it affects the whole system. Hint, you might need this to unlock your machine: # LD_LIBRARY_PATH=/usr/lib rm /var/ld/ld.config
Re: Several problems on Solaris10
On 29/12/2018 13:49, Pierluigi Frullani wrote: I've just upgraded my old Solaris 10 update 8 to Solaris 10 update 11 with the latest patches, but after the reboot with the new update I'm having a lot of problems with dovecot. My version is 2.2.13 ( it was the last one, at the time of the first server setup ). 2.2.13 is from around May 2014. It worked but I can't see why you wouldn't switch to the latest 2.3.4. (You might be seeing what I can't and your question hasn't explained.) I have seen that ( it seems ) the new solaris don't honour the LD_LIBRARY_PATH. I'm sure it does but you shouldn't need it anyway. The first error was a relocation error: relocation error: file /usr/local/dovecot/lib/dovecot/libdovecot-login.so.0: symbol EVP_PKEY_get1_EC_KEY: referenced symbol not found Just to bypass this phase I have linked the openssl-1.0.1e in /usr/local/lib ( libssl and libcrypto ) but then I got a new relocation error: imap: Error: dlopen(/usr/local/dovecot/lib/dovecot/lib20_fts_plugin.so) failed: ld.so.1: imap: fatal: relocation error: file /usr/local/dovecot/lib/dovecot/lib20_fts_plugin.so: symbol http_url_parse: referenced symbol not found Did someone has some idea on how to bypass those troubles ? Did you compile this yourself or are you using someone else's package? Solaris has no files in /usr/local, you must have added those. 1. Do not put your files in /use/local. You will clash with someone else thinking it is the place to put personal stuff. man filesystem: "/opt Root of a subtree for add-on application packages." 2. Do not use LD_LIBRARY_PATH in the run time environment. Instead use the runpath in binaries as set during linking. 3. On configure set the local paths --prefix=/opt/XXX \ --sysconfdir=/etc/opt/XXX \ --localstatedir=/var/opt/XXX \ --mandir=/opt/XXX/share/man \ --docdir=/opt/XXX/share/doc/dovecot \ 4. When linking use -L to point to the libraries. These need not be in the installation location and during build won't be (because you haven't installed yet because you are building new libraries). 5. When linking use -R to point to the installed location of the libraries. It should work if the paths are set correctly in the binaries. A generic package can use $ORIGIN. It's possible libtool is doing its usual trick of making a simple task difficult - I take measures to undo its wrong doing and set -L and -R between libtool and ld (cc -G). I suggest building the new dovecot and getting it right but if you want to diagnose your current installation here are some command hints you might find useful. -- find dovecot: # find /opt -type f -name dovecot /opt/XXX/sbin/dovecot -- is there a package? # pkginfo | grep dovecot # pkginfo | nawk '/dovecot/{print $2}' | while read package do pkginfo -l ${package} done -- what is its run path? # dump -Lv /opt/XXX/sbin/dovecot | grep RUNPATH -- can the runtime linker find the libraries? # ldd -r /opt/XXX/sbin/dovecot
Re: Lda fatal: setgid (102 from userdb lookup) failed
> On 4 Dec 2018, at 5:48 pm, Aki Tuomi wrote: > > On 4.12.2018 3.27, James Brown via dovecot wrote: >> Sorry if this is a nubbie question, but I’m getting: >> >> lda(jlbr...@bordo.com.au)<>: Fatal: setgid(102 >> from userdb lookup) failed with euid=501(jlbrown), gid=20(staff), >> egid=20(staff): Operation not permitted (This binary should probably be >> called with process group set to 102 instead of 20(staff)) >> >> How can I fix this? >> >> (macOS Mojave, Dovecot 2.3.4) >> >> Thanks, >> >> James. > > You need to tell your MTA to execute dovecot-lda as the target user as > you are using per-user UIDs. > > Aki Thanks Aki. It works now. Not sure what it was. In Postfix’s master.cf changed the user in the dovecot line. Also changed dovecot/auth-userdb user and group, and permissions. Anyway, all works now. Thanks again, James.
Lda fatal: setgid (102 from userdb lookup) failed
Sorry if this is a nubbie question, but I’m getting: lda(jlbr...@bordo.com.au)<>: Fatal: setgid(102 from userdb lookup) failed with euid=501(jlbrown), gid=20(staff), egid=20(staff): Operation not permitted (This binary should probably be called with process group set to 102 instead of 20(staff)) How can I fix this? (macOS Mojave, Dovecot 2.3.4) Thanks, James.
"Group doesn't exist: dovecot"
No matter what I do, I always get this error when trying to start Dovecot: $ sudo /usr/local/opt/dovecot/sbin/dovecot Warning: fd limit (ulimit -n) is lower than required under max. load (256 < 1000), because of default_client_limit Fatal: service(stats) Group doesn't exist: dovecot (See service stats { unix_listener /usr/local/var/run/dovecot/stats-writer { group } } setting) The user and group it should be using are _dovecot. Config: $ sudo /usr/local/opt/dovecot/sbin/dovecot -n # 2.3.2.1 (0719df592): /usr/local/etc/dovecot/dovecot.conf # OS: Darwin 18.2.0 x86_64 apfs # Hostname: mailbordocomau-dr.local doveconf: Error: t_readlink(/usr/local/var/run/dovecot/dovecot.conf) failed: readlink() failed: No such file or directory default_internal_user = _dovenull default_login_user = _dovecot first_valid_gid = 0 first_valid_uid = 102 last_valid_gid = 500 last_valid_uid = 105 mail_gid = 102 mail_location = maildir:/usr/local/virtual/%d/%u mail_uid = 102 namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } protocols = imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = _postfix mode = 0660 user = _postfix } unix_listener auth-userdb { mode = 0660 user = _vmail } } ssl_cert =
Re: macOS Mojave: setgroups(501) failed: Too many extra groups
> On 11 Oct 2018, at 7:35 pm, Heiko W. Rupp wrote: > > On Thu, Oct 11, 2018 at 10:55:39AM +0300, Aki Tuomi wrote: > >> Maybe. Have to see when we can implement it though. It could probably >> leverage the min/max_gid setting. > > Actually that was a great hint. > > Setting > last_valid_gid = 100 > in the config and restarting helped. > Having a filter-list instead of fixed upper/lower bounds would be more > flexible. I guess though that in reality most systems/setups have ranges > for different kinds of groups, so that the first/last_valid_gid could > be applied. > > Thanks > Heiko Hi Heiko, I’m also trying to set up Dovecot on a Mojave system. Would you mind leting me know what your settings are? Ie output of: dovecot -n Many thanks, James.
Re: New install - getting error: "Failed to initialize SSL server context: Couldn't parse DH parameters"
Ah… that’s better! No error now. Thanks Aki. > On 13 Nov 2018, at 4:06 pm, Tuomi, Aki wrote: > > Actually you need to use ssl_dh=< > /usr/local/etc/dovecot/dh.pem > > Note the < > > Aki > > Original message > From: "Michael A. Peters" > Date: 13/11/2018 05:44 (GMT+02:00) > To: dovecot@dovecot.org > Subject: Re: New install - getting error: "Failed to initialize SSL server > context: Couldn't parse DH parameters" > > try > > openssl dhparam -out /usr/local/etc/dovecot/dh.pem 2048 > > On 11/12/2018 07:28 PM, James Brown wrote: > > I’m setting up Dovecot using Homebrew on a new server and am getting > > this when I try to login via IMAP: > > > > Nov 13 14:13:35 auth: Debug: auth client connected (pid=30719) > > Nov 13 14:13:35 imap-login: Info: Aborted login (no auth attempts in 0 > > secs): user=<>, rip=::1, lip=::1, secured, > > session= > > Nov 13 14:18:33 auth: Debug: Loading modules from directory: > > /usr/local/Cellar/dovecot/2.3.2.1/lib/dovecot/auth > > Nov 13 14:18:33 auth: Debug: Module loaded: > > /usr/local/Cellar/dovecot/2.3.2.1/lib/dovecot/auth/lib20_auth_var_expand_crypt.so > > Nov 13 14:18:33 auth: Debug: Read auth token secret from > > /usr/local/var/run/dovecot/auth-token-secret.dat > > Nov 13 14:18:33 auth: Debug: auth client connected (pid=30848) > > Nov 13 14:18:33 imap-login: Error: Failed to initialize SSL server > > context: Couldn't parse DH parameters: error:0906D06C:PEM > > routines:PEM_read_bio:no start line: Expecting: DH PARAMETERS: user=<>, > > rip=::1, lip=::1, secured, session= > > Nov 13 14:18:33 imap-login: Info: Disconnected: TLS initialization > > failed. (no auth attempts in 0 secs): user=<>, rip=::1, lip=::1, > > secured, session= > > > > I’ve used: > > > > Openssl gendh 2048 > > > > And put the output: > > > > -BEGIN DH PARAMETERS- > > MIIBCAKCAQEA0IF7kQX32IJFm/5HEVwYf7Be4G9iY86MvLiFLL3wHGqcPT3EMsIv > > YSe5XOT0Q7DGXPOZ+DLlJq8KDHxWKNI6j/0ZaRBrF38CWj8Jqxa8pqo9FVSWj45b > > JwSLqBSoBIEFWibqSE6L8wlV8xjMsB34xLHduJDNbaBzsooN749CopTkmkuGeXKH > > waOEbDzlOq+qHEa4bjx2/e/TnPj0kCrMnfeU4QILo1rJwuN4nY6k7fGwgEDVa2hE > > oOrVfJxyuuuiblahblahblahhhXCGsxhlDQO > > QmzOhHqPovzbBByO9iR5fu3xbNm9YRxPowIBAg== > > -END DH PARAMETERS—— > > > > Into a file dh.pem and then added > > > > ssl_dh=/usr/local/etc/dovecot/dh.pem > > > > To my dovecot.conf file. > > > > Reloaded Dovecot but still get the same error. > > > > Any suggestions? > > > > macOS 10.13.6, Dovecot 2.3.2.1 > > > > Any suggestions? > > > > Thanks, > > > > James. >
Re: New install - getting error: "Failed to initialize SSL server context: Couldn't parse DH parameters"
> On 11/12/2018 07:28 PM, James Brown wrote: >> I’m setting up Dovecot using Homebrew on a new server and am getting this >> when I try to login via IMAP: >> Nov 13 14:13:35 auth: Debug: auth client connected (pid=30719) >> Nov 13 14:13:35 imap-login: Info: Aborted login (no auth attempts in 0 >> secs): user=<>, rip=::1, lip=::1, secured, >> session= >> Nov 13 14:18:33 auth: Debug: Loading modules from directory: >> /usr/local/Cellar/dovecot/2.3.2.1/lib/dovecot/auth >> Nov 13 14:18:33 auth: Debug: Module loaded: >> /usr/local/Cellar/dovecot/2.3.2.1/lib/dovecot/auth/lib20_auth_var_expand_crypt.so >> Nov 13 14:18:33 auth: Debug: Read auth token secret from >> /usr/local/var/run/dovecot/auth-token-secret.dat >> Nov 13 14:18:33 auth: Debug: auth client connected (pid=30848) >> Nov 13 14:18:33 imap-login: Error: Failed to initialize SSL server context: >> Couldn't parse DH parameters: error:0906D06C:PEM routines:PEM_read_bio:no >> start line: Expecting: DH PARAMETERS: user=<>, rip=::1, lip=::1, secured, >> session= >> Nov 13 14:18:33 imap-login: Info: Disconnected: TLS initialization failed. >> (no auth attempts in 0 secs): user=<>, rip=::1, lip=::1, secured, >> session= >> I’ve used: >> Openssl gendh 2048 >> And put the output: >> -BEGIN DH PARAMETERS- >> MIIBCAKCAQEA0IF7kQX32IJFm/5HEVwYf7Be4G9iY86MvLiFLL3wHGqcPT3EMsIv >> YSe5XOT0Q7DGXPOZ+DLlJq8KDHxWKNI6j/0ZaRBrF38CWj8Jqxa8pqo9FVSWj45b >> JwSLqBSoBIEFWibqSE6L8wlV8xjMsB34xLHduJDNbaBzsooN749CopTkmkuGeXKH >> waOEbDzlOq+qHEa4bjx2/e/TnPj0kCrMnfeU4QILo1rJwuN4nY6k7fGwgEDVa2hE >> oOrVfJxyuuuiblahblahblahhhXCGsxhlDQO >> QmzOhHqPovzbBByO9iR5fu3xbNm9YRxPowIBAg== >> -END DH PARAMETERS—— >> Into a file dh.pem and then added >> ssl_dh=/usr/local/etc/dovecot/dh.pem >> To my dovecot.conf file. >> Reloaded Dovecot but still get the same error. >> Any suggestions? >> macOS 10.13.6, Dovecot 2.3.2.1 >> Any suggestions? >> Thanks, >> James. > > > On 13 Nov 2018, at 2:43 pm, Michael A. Peters wrote: > > try > > openssl dhparam -out /usr/local/etc/dovecot/dh.pem 2048 Thanks Michael. Gave that a go, and it successfully created the file, I reloaded Dovecot, but still get the same error. James.
New install - getting error: "Failed to initialize SSL server context: Couldn't parse DH parameters"
I’m setting up Dovecot using Homebrew on a new server and am getting this when I try to login via IMAP: Nov 13 14:13:35 auth: Debug: auth client connected (pid=30719) Nov 13 14:13:35 imap-login: Info: Aborted login (no auth attempts in 0 secs): user=<>, rip=::1, lip=::1, secured, session= Nov 13 14:18:33 auth: Debug: Loading modules from directory: /usr/local/Cellar/dovecot/2.3.2.1/lib/dovecot/auth Nov 13 14:18:33 auth: Debug: Module loaded: /usr/local/Cellar/dovecot/2.3.2.1/lib/dovecot/auth/lib20_auth_var_expand_crypt.so Nov 13 14:18:33 auth: Debug: Read auth token secret from /usr/local/var/run/dovecot/auth-token-secret.dat Nov 13 14:18:33 auth: Debug: auth client connected (pid=30848) Nov 13 14:18:33 imap-login: Error: Failed to initialize SSL server context: Couldn't parse DH parameters: error:0906D06C:PEM routines:PEM_read_bio:no start line: Expecting: DH PARAMETERS: user=<>, rip=::1, lip=::1, secured, session= Nov 13 14:18:33 imap-login: Info: Disconnected: TLS initialization failed. (no auth attempts in 0 secs): user=<>, rip=::1, lip=::1, secured, session= I’ve used: Openssl gendh 2048 And put the output: -BEGIN DH PARAMETERS- MIIBCAKCAQEA0IF7kQX32IJFm/5HEVwYf7Be4G9iY86MvLiFLL3wHGqcPT3EMsIv YSe5XOT0Q7DGXPOZ+DLlJq8KDHxWKNI6j/0ZaRBrF38CWj8Jqxa8pqo9FVSWj45b JwSLqBSoBIEFWibqSE6L8wlV8xjMsB34xLHduJDNbaBzsooN749CopTkmkuGeXKH waOEbDzlOq+qHEa4bjx2/e/TnPj0kCrMnfeU4QILo1rJwuN4nY6k7fGwgEDVa2hE oOrVfJxyuuuiblahblahblahhhXCGsxhlDQO QmzOhHqPovzbBByO9iR5fu3xbNm9YRxPowIBAg== -END DH PARAMETERS—— Into a file dh.pem and then added ssl_dh=/usr/local/etc/dovecot/dh.pem To my dovecot.conf file. Reloaded Dovecot but still get the same error. Any suggestions? macOS 10.13.6, Dovecot 2.3.2.1 Any suggestions? Thanks, James.
Re: Disaster Recovery Help
> On 13 Nov 2018, at 9:48 am, Giovanni Bisanti <mailto:g...@ttyllc.net>> wrote: > > the Account on the iPhone is IMAP, I'm not sure I understand the process you > are describing > iMazing might be able to help you get the mail off the iPhone: https://imazing.com <https://imazing.com/> Then transfer to you Dovecot Maildir. (Have never done this - it’s just a suggestion). James.
Re: macOS Mojave: setgroups(501) failed: Too many extra groups
On 11 Oct 2018, at 5:28 pm, Heiko W. Rupp mailto:h...@pilhuhn.de>> wrote: > > Hello, > > I have recently upgraded to macOS 10.14 (Mojave) and am running into an issue > where one use can no longer log into dovecot via imap. Log shows > > Oct 11 08:10:27 imap(hwr)<12659>: Fatal: > setgroups(501) failed: Too many extra groups > > ... > Any help appreciated > Heiko This came up on the list a few months ago. See thread with subject "dying on osx” on 11 August and 4 September 2018. The only way I managed to get it to work was by changing default_login_user and default_internal_user to my username. As you say, wasn’t an issue with earlier versions of macOS X. Any solution would be most appreciated. James.
Re: v2.3.3 rc1 - Error: sieve: !!BUG!!: Binary compiled from dovecot.sieve is still corrupt
On 01/10/2018 13:37, Stephan Bosch wrote: I'd still like to know whether doing this helps somehow: Does it help when you change the "> 0" at the following code position to "!= 0" ? https://github.com/dovecot/pigeonhole/blob/master/src/lib-sieve/sieve-binary-code.c#L300 --- ../original/src/lib-sieve/sieve-binary-code.c 2018-02-05 19:45:53.0 + +++ src/lib-sieve/sieve-binary-code.c 2018-10-01 14:35:18.553169973 + @@ -297,7 +297,7 @@ return FALSE; /* Read first integer bytes [1xxx] */ - while ( (ADDR_DATA_AT(address) & 0x80) > 0 ) { + while ( (ADDR_DATA_AT(address) & 0x80) != 0 ) { if ( ADDR_BYTES_LEFT(address) > 0 && bits > 0) { integer |= ADDR_DATA_AT(address) & 0x7F; ADDR_JUMP(address, 1); Quick reply: It does. I can't see why. James.
Re: v2.3.3 rc1 - Error: sieve: !!BUG!!: Binary compiled from dovecot.sieve is still corrupt
On 25/09/2018 22:39, Stephan Bosch wrote: Hello, Something mightily weird is going on at your end. It doesn't fail here This is correct. I believe there to be a compiler problem. Removing optimisation from sieve-binary-code.c gives success. Thank you to Stephan Bosch for testing this for me and pointing me in the right direction. My debugging was confused because dovecot flags are passed to pigeonhole, eg $DOVECOT_CFLAGS. This is why the build of dovecot affects pigeonhole. Once I removed these I could control the pigeonhole build and I was able to isolate the problem. James.
Re: v2.3.3 rc1 - Error: sieve: !!BUG!!: Binary compiled from dovecot.sieve is still corrupt
On 28/09/2018 12:38, Aki Tuomi wrote: It was studio cc. gcc doesn't make it through configure and I didn't ask why. Can you share a little bit more info on how did the compile (or configure even) fail with gcc on Solaris 11? $ ./configure $ARGS ... checking Linux compatible mremap()... no checking whether shared mmaps get updated by write()s... no checking whether fd passing works... no configure: error: fd passing is required for Dovecot to work Which in the log corresponds to: configure:22685: ./conftest ./configure[2026]: eval: line 1: 22335: Memory fault(coredump) Appears to the option "-mfunction-return=thunk" that cause the problem, remove and no core dump. If you thunk that was all think again. My gcc builds fail to link ending with lots of undefined symbols. It is probably libtool obstructing portability but I have an easy solution of using another compiler and ignoring libtool. You can use --disable-harderning to disable these things. --disable-hardening
Re: v2.3.3 rc1 - Error: sieve: !!BUG!!: Binary compiled from dovecot.sieve is still corrupt
On 27/09/2018 16:14, Sami Ketola wrote: It was studio cc. gcc doesn't make it through configure and I didn't ask why. Can you share a little bit more info on how did the compile (or configure even) fail with gcc on Solaris 11? $ ./configure $ARGS ... checking Linux compatible mremap()... no checking whether shared mmaps get updated by write()s... no checking whether fd passing works... no configure: error: fd passing is required for Dovecot to work Which in the log corresponds to: configure:22685: ./conftest ./configure[2026]: eval: line 1: 22335: Memory fault(coredump) Appears to the option "-mfunction-return=thunk" that cause the problem, remove and no core dump. Older gccs do not have -mfunction-return. as I have no problems in compiling dovecot and pigeonhole on my Solaris 11.3 system with gcc. The version that ships with my Solaris is 4.5.2. Strictly speaking Solaris 11 does not ship with gcc, one can install it [from the OS vendor] with pkg and there is a choice of versions. # pkg list -a | grep gcc-c I have gcc versions installed: 4.9.5, 5.5.0, 6.4.0, 7.3.0 and 8.2.0. I also have Sun Studio 12.5 installed but I have not even tried to compile dovecot wit that yet. Current Release - Oracle Developer Studio 12.6. James.
Re: v2.3.3 rc1 - Error: sieve: !!BUG!!: Binary compiled from dovecot.sieve is still corrupt
On 27/09/2018 14:55, Josef 'Jeff' Sipek wrote: On Thu, Sep 27, 2018 at 10:42:16 +0100, James wrote: $ dovecot -c dovecot.conf -n # 2.3.3.rc1 (14e4920d8): dovecot.conf # Pigeonhole version 0.5.2 (7704de5e) # OS: SunOS 5.11 i86pc Out of curiosity, is this a Solaris system or an illumos system? Not Illumos. Test builds on 10 and 11.3 - RC exercise, I'll add Sparc when my machine is powered on. Tested run on Solaris 10 and 11.3 - RC exercise. Reports from 11.3, results so far the same either way. Packages built on Solaris 10 as LCD unless there is good reason to have a package per rev.
Re: v2.3.3 rc1 - Error: sieve: !!BUG!!: Binary compiled from dovecot.sieve is still corrupt
On 27/09/2018 13:40, Stephan Bosch wrote: Address Line Code : DEBUG BLOCK: 3 0001: EXTENSIONS [1]: 0002: vacation 0004:2: VACATION 0007:4: seconds: NUM 5 0009: Binary is corrupt. The line numbers differs and 86400 is read as 5. It is like it has forgotten the size of an integer or is confused about endianness. There is something strange, like an #if that guesses wrong. At least I have somewhere to start looking. Thank you for checking at your end, I was worried the RC had introduced an error and your result suggests not. RCs are for testing and I am. The number is stored as a chain of bytes of which the most significant bit indicates whether the next byte still belongs to the number. If this bit is somehow interpreted wrong, the first byte of this number would read as 5, thereby returning '5' as the result and ignoring subsequent bytes (causing corruption at the next item to read). Since you're using SunOS, your compiler may be doing something funky. Which compiler is used anyway? Perhaps different versions for the Dovecot releases that do and don't work? It was studio cc. gcc doesn't make it through configure and I didn't ask why. I have some other things to do but will look at this again later. Thank you for the byte code explanations. The coding at this point is hard to follow with the pointers-to-functions and #defines. James.
Re: v2.3.3 rc1 - Error: sieve: !!BUG!!: Binary compiled from dovecot.sieve is still corrupt
On 25/09/2018 22:39, Stephan Bosch wrote: Something mightily weird is going on at your end. It doesn't fail here (see below). First of all, what is your configuration (output from `dovecot -n`)? You have dovecot.conf but... $ dovecot -c dovecot.conf -n # 2.3.3.rc1 (14e4920d8): dovecot.conf # Pigeonhole version 0.5.2 (7704de5e) # OS: SunOS 5.11 i86pc # Hostname: mailhost doveconf: Warning: please set ssl_dh=doveconf: Warning: You can generate it with: dd if=/var/opt/xxx/lib/dovecot/ssl-parameters.dat bs=1 skip=88 | openssl dhparam -inform der > /etc/opt/xxx/dovecot/dh.pem mail_debug = yes mail_gid = staff mail_location = maildir:/path/to/%d/%n/Maildir mail_uid = james managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext postmaster_address = postmas...@domain.tld ssl_dh = # hidden, use -P to show it Also, can you make a hex dump of the binary (using `sieve-dump -h .svbin`). As said the svbin is identical to the one create by the previous version. Comparing the dump: Block 0 differs because it has the source file name. Block 1 is identical Block 2 is identical. It is this block that is declared corrupt. Finally, can you try to explicitly delete the binary (preferably after preserving it elsewhere) so that it is guaranteed to be created fresh? I did each time. ## Success at my end: Spot the difference... Address Line Code : DEBUG BLOCK: 3 0001: EXTENSIONS [1]: 0002: vacation 0004:3: VACATION 0007:5: seconds: NUM 86400 Address Line Code : DEBUG BLOCK: 3 0001: EXTENSIONS [1]: 0002: vacation 0004:2: VACATION 0007:4: seconds: NUM 5 0009: Binary is corrupt. The line numbers differs and 86400 is read as 5. It is like it has forgotten the size of an integer or is confused about endianness. There is something strange, like an #if that guesses wrong. At least I have somewhere to start looking. Thank you for checking at your end, I was worried the RC had introduced an error and your result suggests not. RCs are for testing and I am.
Re: v2.3.3 rc1 - Error: sieve: !!BUG!!: Binary compiled from dovecot.sieve is still corrupt
On 24/09/2018 13:35, Stephan Bosch wrote: You can enable `-d -' and `-t - -Tlevel=matching' as well. $ sieve-test -D -d - -t - -Tlevel=matching -c dovecot.conf sieve message sieve-test(james): Debug: Effective uid=1001, gid=10, home=/home/james sieve-test(james): Debug: maildir++: root=/path/to//james/Maildir, index=, indexpvt=, control=, inbox=/path/to//james/Maildir, alt= sieve-test(james): Debug: sieve: Pigeonhole version 0.5.2 (7704de5e) initializing sieve-test(james): Debug: sieve: include: sieve_global is not set; it is currently not possible to include `:global' scripts. debug: file storage: Using Sieve script path: /home/james/sieve. debug: file script: Opened script `sieve' from `/home/james/sieve'. debug: Script `sieve' from /home/james/sieve successfully compiled. * Script metadata (block: 0): class = file class.version = 0 location = /home/james/sieve * Required extensions (block: 1): 0: vacation (id: 9) * Main program (block: 2): Address Line Code : DEBUG BLOCK: 3 0001: EXTENSIONS [1]: 0002: vacation 0004:2: VACATION 0007:4: seconds: NUM 5 0009: Binary is corrupt. ## Started executing script 'sieve' VACATION: #ERROR#: unknown optional operand 2: [[EXECUTION ABORTED]] ## Finished executing script 'sieve' sieve-test(james): Info: corrupt binary deleted. sieve-test(james): Error: unlink((null)) failed: Bad address (in sieve-test.c:426) sieve-test(james): Info: final result: failed; resolved with successful implicit keep