Re: Minor patches for builds against ancient platforms
Quoting Joseph Tam: > If this is output on the dovecot server itself so there's no mismatch > in pathnames. Have you checked whether the dovecot user can traverse > all the way from / to /u2/usermail/luser/ The dovecot user, as in the "dummy" user dovecot uses for sandboxing, or the UID of the user logged in via IMAP through dovecot? No, the (dovecot user) doesn't have access to the director(ies), but the logged-in users DO. That's no different from the mail directories in /home/*, though, which are 0700/owned-by-their-respective-users. I have confirmed by using "su -" to the various UIDs that they can fully access the mailboxes behind the symlinked directories. > I'm thinking no .subscription would be better. Done, but it makes no difference. > Dovecot does have chroot-ing stuff that might impede symlink following: > > https://wiki.dovecot.org/Chrooting I'm not running dovecot chrooted. That's a bear of a very different species, and one I'd not care to wrestle for this sort of setup. Maybe in a "vpopmail" type of situation where dovecot only runs as a delegate UID where there are no real system UIDs/GIDs for the users in question. There's no dovecotian "AllowSymlink" analogue to Apache's FollowSymLinks directive, I assume? I scoured through the documentation, but didn't see anything, but it's not the first time I've missed things in documentation. > It seems you have more basic file access problems. I suspect so, but it's a strange one, because (al)pine and UW-imapd have accessed these mailboxes without any issues for many years, as much as it comes as a shock that such decrepit software could ever be accused of performing correctly. > Nothing with verbose logging (set mail_debug = yes)? Try the simple case > > ln -s /u2/usermail/luser/OLD_INBOX/INBOX_2016_01_to_08 > ~luser/mail/testbox Will do. My thanks, Joseph. =M=
Re: Minor patches for builds against ancient platforms
I've tried changing how I symbolically linked the mailboxes, i.e., creating a sub-directory that is symlinked into the user's mail/ directory versus symbolically linking the mbox files themselves, etc. No dice. Permissions are fine. I've even resorted to changing the index locking strategy, to no avail. I've tested my setup by symlinking both folders and boxes with user/0700 permissions, and they all work fine. Folders that are NOT symbolically linked work perfectly, and have various levels of hierarchy that are selectable as expected. Nothing appears in the logs. $ cd ~/mail $ ls -l -rw--- 1 2411625 Dec 16 09:12 Dovecot lrwxrwxrwx 1 21 Jun 13 18:01 OldMail -> /u2/usermail/luser -rwx-- 8 4096 Jan 1 12:09 "Open Source Projects" If this is output on the dovecot server itself so there's no mismatch in pathnames. Have you checked whether the dovecot user can traverse all the way from / to /u2/usermail/luser/ I've (rm ~/mail/.subscriptions && touch ~/mail/.subscriptions) to flush any subscriptions file issues. I'm thinking no .subscription would be better. Is there a subtle interaction with mail_full_filesystem_access settings, or similar that might be getting in the way? Dovecot does have chroot-ing stuff that might impede symlink following: https://wiki.dovecot.org/Chrooting Other data: there are fs quotas on / but not /u2. That shouldn't matter, but I will concede that I'm not a little ignorant about such things. It seems you have more basic file access problems. How might I go about further debugging this? I've tried to manually doveadm index those mailboxes, which doesn't give me any errors, but it also returns far too quickly to give me the impression that it's done anything. Same result. Nothing with verbose logging (set mail_debug = yes)? Try the simple case ln -s /u2/usermail/luser/OLD_INBOX/INBOX_2016_01_to_08 ~luser/mail/testbox then $ telnet localhost 10143 x0 login luser *** x1 SELECT testbox What do you get here and in your logs? If this doesn't produce any usable diagnostics, I would pull out the heavy duty process trace tool and trace the imap process to figure out what it's really doing and where it's failing. Joseph Tam
AW: Need Help to analyze the error or is it a bug?
Hello, thank You for helping me. I did not remember the version before the update. I use FreeBSD 11.0. The OS tells me, from time to time, there are updates to do. So it was and I have also compiled dovecot via the FreeBSD ports system with some others. These are my settings: [/usr/ports/mail/dovecot2] => make showconfig ===> The following configuration options are available for dovecot2-2.2.30.2: DOCS =on: Build and/or install documentation EXAMPLES =on: Build and/or install examples KQUEUE =on: kqueue(2) support LIBWRAP=on: TCP wrapper support LZ4=off: LZ4 compression support VPOPMAIL =off: vpopmail support > Database support CDB=off: CDB database support LDAP =on: LDAP protocol support MYSQL =on: MySQL database support PGSQL =off: PostgreSQL database support SQLITE =off: SQLite database support > Full text search plugins ICU=off: Use libicu for FTS unicode normalization LUCENE =off: CLucene FTS support SOLR =off: Solr FTS support TEXTCAT=off: Libtextcat FTS support > GSSAPI Security API support: you have to select exactly one of them GSSAPI_NONE=off: Build without GSSAPI support GSSAPI_BASE=off: Use GSSAPI from base GSSAPI_HEIMDAL=off: Use Heimdal GSSAPI from security/heimdal GSSAPI_MIT =on: Use MIT GSSAPI from security/krb5 Today the error has gone. I DO NOT know why. Yesterday I had make some more other updates for i.e. for llvm... + openjdk7 with # portupgrade -r ( r means recursive). It takes very long time and the terminal-session was broken so the connection was lost. Today I come back and take a look. it seems that it has finished and I go on to find out the dovecot error. I do then: #gdb --args /usr/local/libexec/dovecot/imap -u @domain.tld But "auth" like this: " Error: auth: environment corrupt; missing value " was no more broken. But there was some rights not ok for the users db. I changed this and try again #gdb --args /usr/local/libexec/dovecot/imap -u @domain.tld wow: I get a login. Then without changing any settings in dovecot I try the Outlook client .and it works ??? I don't know what it was. I could not figure out what was the authorize problem because of to less debug information. When I have not program this software an error message like this "Fatal: Couldn't connect to auth socket" is not very helpful, also there are in /var/run/dovecot 7 auth sockets and more then 38 sockets in all ?! If you have an idea, how I could better debug this please tell me. But again, thank for trying helping me. Harald -Ursprüngliche Nachricht- Von: dovecot [mailto:dovecot-boun...@dovecot.org] Im Auftrag von Teemu Huovila Gesendet: Freitag, 16. Juni 2017 09:19 An: dovecot@dovecot.org Betreff: Re: Need Help to analyze the error or is it a bug? On 15.06.2017 01:45, Dipl.-Ing. Harald E. Langner wrote: > After done an update to dovecot-2.2.30.2 > > my connection is broken since days. > > all what I try every time the same error: > > Jun 15 00:02:18 auth: Error: auth: environment corrupt; missing value > for DOVECOT_ Jun 15 00:02:18 auth: Fatal: unsetenv(RESTRICT_SETUID) > failed: Bad address Jun 15 00:02:18 master: Error: service(auth): > command startup failed, throttling for 2 secs Could you post the output of "doveconf -n". Please also describe from which version you upgraded to v2.2.30.2 and how you did the upgrade? Are you compiling Dovecot yourself? What are the configuration & compilation options etc. br, Teemu > > > I try this: > > # doveadm -Dv auth test -x service=imap theusername mypassword > > output: > > Debug: Loading modules from directory: /usr/local/lib/dovecot > Debug: Module loaded: /usr/local/lib/dovecot/lib20_virtual_plugin.so > Debug: Loading modules from directory: /usr/local/lib/dovecot/doveadm > Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: > /usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: Undefined > symbol "acl_user_module" (this is usually intentional, so just ignore > this message) > Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: > /usr/local/lib/dovecot/doveadm/lib10_doveadm_expire_plugin.so: > Undefined symbol "expire_set_lookup" (this is usually intentional, so > just ignore this message) > Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: > /usr/local/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so: > Undefined symbol "quota_user_module" (this is usually intentional, so > just ignore this message) > Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: > /usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so: Undefined > symbol "fts_filter_filter" (this is usually intentional, so just > ignore this message) > Debug: Skipping module
Error: fts_solr: received invalid uid '0'?
I'm getting a lot of: Jun 16 11:40:40 thebighonker dovecot: imap(ler/58755): Error: fts_solr: received invalid uid '0' Jun 16 11:40:40 thebighonker dovecot: imap(ler/58755): Error: fts_solr: received invalid uid '0' in my logs. What can I supply to help get to the bottom of it? doveconf -n: # 2.2.30.2 (c0c463e): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.18 (29cc74d) # OS: FreeBSD 11.1-BETA1 amd64 auth_default_realm = lerctr.org auth_mechanisms = plain login auth_realms = lerctr.org thebighonker.lerctr.org tbh.lerctr.org auth_username_format = %Ln default_vsz_limit = 1 G deliver_log_format = msgid=%m: %$ (subject=%s from=%f size=%w) lda_mailbox_autocreate = yes listen = 192.147.25.65, :: lmtp_save_to_detail_mailbox = yes login_access_sockets = tcpwrap mail_attribute_dict = file:%h/mail/.imap/dovecot-mail-attributes mail_debug = yes mail_location = mbox:~/mail:INBOX=~/mail/INBOX mail_log_prefix = "%s(%u/%p): " mail_plugins = " fts fts_solr notify stats virtual" mail_privileged_group = mail mail_server_admin = mailto:l...@lerctr.org mail_server_comment = LERCTR Mail Server mailbox_list_index = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext vacation-seconds editheader mboxmetadata servermetadata imapsieve vnd.dovecot.imapsieve namespace archive { hidden = no list = no location = mbox:~/MAIL-ARCHIVE prefix = ARCHIVE/ separator = / } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox INBOX { auto = create } mailbox SENT { special_use = \Sent } mailbox SPAM { special_use = \Junk } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } mailbox virtual/Flagged { special_use = \Flagged } mailbox virtual/all { special_use = \All } prefix = } namespace virtual { hidden = no list = yes location = virtual:~/MAIL-VIRTUAL:INDEX=MEMORY prefix = Virtual/ separator = / } passdb { args = failure_show_msg=yes session=yes max_requests=20 driver = pam } plugin { fts = solr fts_autoindex = yes fts_solr = url=http://thebighonker.lerctr.org:8983/solr/dovecot/ fts_tika = http://localhost:9998/tika/ imapsieve_mailbox1_before = file:/usr/local/share/dovecot-pigeonhole/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = SPAM imapsieve_mailbox2_before = file:/usr/local/share/dovecot-pigeonhole/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = SPAM imapsieve_mailbox2_name = * imapsieve_url = sieve://thebighonker.lerctr.org mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change append mail_log_fields = uid box msgid size from subject vsize flags recipient_delimiter = + sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_execute_bin_dir = /usr/local/share/dovecot-pigeonhole/sieve sieve_extensions = +editheader +vacation-seconds +mboxmetadata +servermetadata sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.execute sieve_pipe_bin_dir = /usr/local/share/dovecot-pigeonhole/sieve sieve_plugins = sieve_imapsieve sieve_extprograms stats_command_min_time = 1 mins stats_domain_min_time = 12 hours stats_ip_min_time = 12 hours stats_memory_limit = 16 M stats_refresh = 5s stats_session_min_time = 15 mins stats_track_cmds = yes stats_user_min_time = 1 hours } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-client { mode = 0666 } unix_listener auth-master { mode = 0666 } } service indexer-worker { drop_priv_before_exec = yes } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } service stats { chroot = empty client_limit = 0 drop_priv_before_exec = no executable = stats extra_groups = fifo_listener stats-mail { group = mode = 0666 user = } fifo_listener stats-user { group = mode = 0666 user = } group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener stats { group = mode = 0666 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service tcpwrap { unix_listener login/tcpwrap { group = $default_login_user mode = 0600 user = $default_login_user } } ssl_cert = http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: larry...@gmail.com US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281
doveadm-server core dump
Hello, coredump found after a force-resync and a replicate Emmanuel. (dovecot-ee 2.2.30.2-1) [New LWP 17168] Core was generated by `dovecot/doveadm-server'. Program terminated with signal SIGABRT, Aborted. #0 0x7fa334127c37 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 #0 0x7fa334127c37 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 resultvar = 0 pid = 17168 selftid = 17168 #1 0x7fa33412b028 in __GI_abort () at abort.c:89 save_stage = 2 act = {__sigaction_handler = {sa_handler = 0x20d2b48, sa_sigaction = 0x20d2b48}, sa_mask = {__val = {34255104, 0, 140338942616871, 1, 0, 181, 140338929827120, 140725975364353, 0, 0, 140338942645461, 0, 140338930777984, 140338937001888, 34203576, 140725975364140}}, sa_flags = 0, sa_restorer = 0x7ffd51c5b1c8} sigs = {__val = {32, 0 }} #2 0x7fa334546766 in default_fatal_finish (type=, status=status@entry=0) at failures.c:201 backtrace = 0x209df68 "/usr/lib/dovecot/libdovecot.so.0(+0x8d770) [0x7fa334546770] -> /usr/lib/dovecot/libdovecot.so.0(+0x8d84e) [0x7fa33454684e] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fa3344dcffb] -> dovecot/do"... #3 0x7fa33454684e in i_internal_fatal_handler (ctx=0x7ffd51c5b3b0, format=, args=) at failures.c:670 status = 0 #4 0x7fa3344dcffb in i_panic (format=format@entry=0x4530f8 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:275 ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0, timestamp_usecs = 0} args = {{gp_offset = 40, fp_offset = 48, overflow_arg_area = 0x7ffd51c5b4a0, reg_save_area = 0x7ffd51c5b3e0}} #5 0x0043a1eb in dsync_brain_sync_mailbox_deinit (brain=brain@entry=0x20d2b48) at dsync-brain-mailbox.c:371 last_common_uid = 0 last_messages_count = 0 last_common_modseq = 4428091 changes_during_sync = 0x0 last_common_pvt_modseq = 140725975364904 require_full_resync = false error = MAIL_ERROR_NONE __FUNCTION__ = "dsync_brain_sync_mailbox_deinit" #6 0x0043adc6 in dsync_brain_slave_recv_mailbox (brain=brain@entry=0x20d2b48) at dsync-brain-mailbox.c:841 dsync_box = 0x20d0878 local_dsync_box = {mailbox_guid = "\344B\277\017j\022\271T\002\253\000\000\367\362\265", , mailbox_lost = false, have_guids = true, have_save_guids = true, have_only_guid128 = false, uid_validity = 1497621508, uid_next = 10, messages_count = 9, first_recent_uid = 1, highest_modseq = 3, highest_pvt_modseq = 0, cache_fields = {arr = {buffer = 0x209d208, element_size = 24}, v = 0x209d208, v_modifiable = 0x209d208}} box = 0x20e0f88 errstr = 0x2f resync_reason = 0x466b58 "UIDVALIDITY changed during a stateful sync, need to restart" error = MAIL_ERROR_NONE ret = resync = 255 __FUNCTION__ = "dsync_brain_slave_recv_mailbox" #7 0x004387db in dsync_brain_run_real (changed_r=0x7ffd51c5b67f, brain=0x20d2b48) at dsync-brain.c:651 ret = true orig_state = DSYNC_STATE_SLAVE_RECV_MAILBOX orig_box_recv_state = DSYNC_BOX_STATE_MAILBOX orig_box_send_state = DSYNC_BOX_STATE_MAILBOX changed = false #8 dsync_brain_run (brain=brain@entry=0x20d2b48, changed_r=changed_r@entry=0x7ffd51c5b67f) at dsync-brain.c:687 _data_stack_cur_id = 5 #9 0x00438ae1 in dsync_brain_run_io (context=0x20d2b48) at dsync-brain.c:110 brain = 0x20d2b48 changed = false try_pending = true #10 0x0044cc7f in dsync_ibc_stream_input (ibc=0x20c1a10) at dsync-ibc-stream.c:230 ibc = 0x20c1a10 #11 0x7fa33455a802 in io_loop_call_io (io=0x20becc0) at ioloop.c:599 ioloop = 0x20a9f20 t_id = 4 __FUNCTION__ = "io_loop_call_io" #12 0x7fa33455bd47 in io_loop_handler_run_internal (ioloop=ioloop@entry=0x20a9f20) at ioloop-epoll.c:223 ctx = 0x20ab100 list = 0x20c14f0 io = tv = {tv_sec = 4, tv_usec = 12} events_count = msecs = ret = 1 i = 0 call = __FUNCTION__ = "io_loop_handler_run_internal" #13 0x7fa33455a89c in io_loop_handler_run (ioloop=ioloop@entry=0x20a9f20) at ioloop.c:648 No locals. #14 0x7fa33455aa58 in io_loop_run (ioloop=0x20a9f20) at ioloop.c:623 __FUNCTION__ = "io_loop_run" #15 0x0042065e in cmd_dsync_server_run (_ctx=0x20b6168, user=) at doveadm-dsync.c:1169 ctx = 0x20b6168 ibc = 0x20c1a10 brain = 0x20d2b48 temp_prefix = 0x209cb68 state_str = 0x0 sync_type = name = 0x20a9f00 "10.33.207.136" process_title_prefix = 0x209cb40 "10.33.207.136 " mail_error = MAIL_ERROR_NONE #16 0x00421f96 in doveadm_mail_next_user (ctx=ctx@entry=0x20b6168,
10-ssl ssl = no but dovecot still reads certs
I'm bringing up a new email server starting without TLS initially. In 10-ssl.conf I set ssl = no, but the default ssl_cert and ssl_key lines are not commented out. I got the obvious error message: -- doveconf: Fatal: Error in configuration file /usr/local/etc/dovecot/conf.d/10-ssl.conf line 12: ssl_cert: Can't open file /etc/ssl/certs/dovecot.pem: No such file or directory /usr/local/etc/rc.d/dovecot: WARNING: failed to start dovecot -- No big deal, but I don't remember this being an issue the last time I set up a server. You would think if ssl=no, the ssl_cert and ssl_key files would not be opened.
Re: Need Help to analyze the error or is it a bug?
On 15.06.2017 01:45, Dipl.-Ing. Harald E. Langner wrote: > After done an update to dovecot-2.2.30.2 > > my connection is broken since days. > > all what I try every time the same error: > > Jun 15 00:02:18 auth: Error: auth: environment corrupt; missing value for > DOVECOT_ > Jun 15 00:02:18 auth: Fatal: unsetenv(RESTRICT_SETUID) failed: Bad address > Jun 15 00:02:18 master: Error: service(auth): command startup failed, > throttling for 2 secs Could you post the output of "doveconf -n". Please also describe from which version you upgraded to v2.2.30.2 and how you did the upgrade? Are you compiling Dovecot yourself? What are the configuration & compilation options etc. br, Teemu > > > I try this: > > # doveadm -Dv auth test -x service=imap theusername mypassword > > output: > > Debug: Loading modules from directory: /usr/local/lib/dovecot > Debug: Module loaded: /usr/local/lib/dovecot/lib20_virtual_plugin.so > Debug: Loading modules from directory: /usr/local/lib/dovecot/doveadm > Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: > /usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: Undefined symbol > "acl_user_module" (this is usually intentional, so just ignore this message) > Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: > /usr/local/lib/dovecot/doveadm/lib10_doveadm_expire_plugin.so: Undefined > symbol "expire_set_lookup" (this is usually intentional, so just ignore this > message) > Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: > /usr/local/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so: Undefined > symbol "quota_user_module" (this is usually intentional, so just ignore this > message) > Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: > /usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so: Undefined symbol > "fts_filter_filter" (this is usually intentional, so just ignore this message) > Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: > /usr/local/lib/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so: Undefined > symbol "mail_crypt_user_get_public_key" (this is usually intentional, so just > ignore this message) > Error: Timeout waiting for handshake from auth server. my pid=63521, input > bytes=0 > Fatal: Couldn't connect to auth socket > > > dlopen() failed, Undefined symbols ... Is this a bug? > > How do I check what is going wrong with this auth socket error? > > > 2nd) I'm using > > dovecot.conf > protocols = imap lmtp > !include conf.d/*.conf > > 10-auth.conf , > > 10-master.conf, > > auth-passwdfile.conf.ext, > > 10-ssl.conf > > and 10-logging.conf (all logs are on) > > All others, form the 29 configuration files, I haven't touch. It has worked, > before I have done the update. What has changed? For what in the *.conf > should I locking for? > > > Thanks a lot.