Case sensitive problem for vacation with dovecot lmtp pigeonhole
Hi list, This issue was raised here by John Fawcett on 12/02/19 but got no response. So, as I'm facing the same issue, I would like to try again: When using the vacation sieve extension, if I send an email to user.n...@mydomain.com it will work, but not to user.n...@mydomain.com. Is there a way to make the user part case insensitive so that the vacation auto-reply is fired whatever the way user part is written ? doveconf -n output: # 2.3.5.1 (7ec6d0ade): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.5.5 (2483b085) # OS: Linux 3.10.0-957.5.1.el7.x86_64 x86_64 CentOS Linux release 7.6.1810 (Core) ext4 # Hostname: myserver.mydomain.com auth_master_user_separator = * auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@* auth_verbose = yes debug_log_path = /var/log/dovecot-debug.log default_client_limit = 10640 default_process_limit = 2128 default_vsz_limit = 0 hostname = mydomain.com info_log_path = /var/log/dovecot.log lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = * log_path = /var/log/dovecot-error.log login_greeting = Welcome mail_gid = vmail mail_location = maildir:/var/vmail/%Ln mail_plugins = quota fts fts_lucene mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = maildir:/var/vmail/%Ln mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = type = private } passdb { args = /usr/local/etc/dovecot/deny-users deny = yes driver = passwd-file } passdb { args = /usr/local/etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { fts = lucene fts_autoindex = yes fts_lucene = whitespace_chars=@. quota = maildir:User quota quota_rule = *:storage=30GB quota_rule2 = Trash:storage=+200M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = /var/sieve-scripts/%Ln.sieve sieve_dir = /var/vmail/%Ln/home/sieve } postmaster_address = postmas...@mydomain.com protocols = imap pop3 lmtp sieve sieve quota_full_tempfail = yes service auth-worker { user = $default_internal_user } service auth { unix_listener /var/spool/postfix/private/auth { mode = 0777 user = vmail } } service imap-login { inet_listener imaps { port = 993 ssl = yes } process_min_avail = 10 } service imap { process_limit = 2128 } service lmtp { executable = lmtp -L process_min_avail = 15 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3s { port = 995 ssl = yes } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh user = nobody } ssl_cert =
Re: Status of SMTPUTF8?
I necro-bump this thread as I have the same problem since I switched to LMTP from LDA (as the wiki recommend). Any news to make dovecot LMTP postix compliant ? Le 08/11/2016 à 17:13, Noah Tilton a écrit : I was wondering whether there is a roadmap for adding SMTPUTF8 support to Dovecot? My delivery pattern is Postfix -> Dovecot LMTP and it is choking on utf8 messages. I might be able to volunteer some of my time as a developer. Another thread about this seemed to go unanswered: http://dovecot.org/list/dovecot/2016-September/105474.html http://unix.stackexchange.com/questions/320091/configure-postfix-and-dovecot-lmtp-to-receive-mail-via-smtputf8 -Noah
Bug: Dovecot index loosing sync with FTS despite "fts_autoindex = yes"
Le 21/05/2018 à 12:38, Aki Tuomi a écrit : > can you try turning on pluign { fts_enforced = yes } and repeat your test? Same (wrong) result: 1. Send an email with "too6Ouka" in the body 2. Search against "too6Ouka": # doveadm search -u username mailbox INBOX body too6Ouka --> No result 3. Force re-index: # doveadm fts rescan -u username 4. Search again against "too6Ouka": # doveadm search -u username mailbox INBOX body too6Ouka --> e09cce0283e8695ab76002deed92 29055 Don't know if relevant, but on a side note, if I send a second message with "too6Ouka" in the body, followed by: # doveadm -v index -u username Inbox --> doveadm(username): Info: INBOX: Cache is already up to date And a search against the pattern immediately return only one result instead of two. # doveconf -n # 2.2.34 (874deae): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.22 (22940fb7) # OS: Linux 3.10.0-862.2.3.el7.x86_64 x86_64 CentOS Linux release 7.5.1804 (Core) ext4 # Hostname: mailserver.domain auth_master_user_separator = * auth_socket_path = /var/run/dovecot/auth-userdb auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@* auth_verbose = yes base_dir = /var/run/dovecot/ debug_log_path = /var/log/dovecot-debug.log default_client_limit = 5120 default_process_limit = 1024 default_vsz_limit = 2 G hostname = domain info_log_path = /var/log/dovecot.log lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = * log_path = /var/log/dovecot-error.log login_greeting = Welcome to dovecot on mydomain login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c mail_gid = vmail mail_location = maildir:/var/vmail/%Ln mail_plugins = " quota stats fts fts_solr" mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace { inbox = yes location = maildir:/var/vmail/%Ln mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = type = private } passdb { args = /usr/local/etc/dovecot/deny-users deny = yes driver = passwd-file } passdb { args = /usr/local/etc/dovecot/master-users driver = passwd-file master = yes } passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { fts = solr fts_autoindex = yes fts_enforced = yes fts_solr = url=http://localhost:8983/solr/dovecot/ break-imap-search quota = maildir:User quota quota_rule = *:storage=20GB quota_rule2 = Trash:storage=+200M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = /var/sieve-scripts/%Ln.sieve sieve_dir = /var/vmail/%Ln/home/sieve stats_command_min_time = 1 mins stats_domain_min_time = 12 hours stats_ip_min_time = 12 hours stats_memory_limit = 16 M stats_refresh = 30 secs stats_session_min_time = 15 mins stats_track_cmds = yes stats_user_min_time = 1 hours } postmaster_address = postmaster@domain protocols = imap pop3 sieve sieve service auth-worker { user = root } service auth { unix_listener /var/spool/postfix/private/auth { mode = 0777 user = vmail } } service imap-login { inet_listener imaps { port = 993 ssl = yes } process_min_avail = 10 } service imap { process_limit = 2128 } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3s { port = 995 ssl = yes } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh user = nobody } service stats { fifo_listener stats-mail { mode = 0600 user = vmail } } ssl_cert = > Aki > > > On 21.05.2018 13:31, kada...@gmail.com wrote: >> Le 21/05/2018 à 11:46, Aki Tuomi a écrit : >>> That message is coming from the temporary raw user, not from the actual >>> user. >>> >>> How did you verify that the FTS indexes are not created or updated >>> automatically? >> I send an email containing a specific strings ('thahB8ea' in that case) >> and search agains it: >> # doveadm search -u username mailbox INBOX body thahB8ea >> --> No result >> (same with integrated search on roundcube) >> >> Then I force a rescan, followed by the same search: >> # doveadm fts rescan -u username >> # doveadm search -u username mailbox INBOX body thahB8ea >> (Long waiting time) >> --> e09cce0283e8695ab76002deed92 29048 >> >> So I conclude that autoindexing is not working. >> >> Thanks, >> kfx >>
Re: Bug: Dovecot index loosing sync with FTS despite "fts_autoindex = yes"
Le 21/05/2018 à 11:46, Aki Tuomi a écrit : > That message is coming from the temporary raw user, not from the actual > user. > > How did you verify that the FTS indexes are not created or updated > automatically? I send an email containing a specific strings ('thahB8ea' in that case) and search agains it: # doveadm search -u username mailbox INBOX body thahB8ea --> No result (same with integrated search on roundcube) Then I force a rescan, followed by the same search: # doveadm fts rescan -u username # doveadm search -u username mailbox INBOX body thahB8ea (Long waiting time) --> e09cce0283e8695ab76002deed92 29048 So I conclude that autoindexing is not working. Thanks, kfx > > Aki > > > On 04.05.2018 12:47, kada...@gmail.com wrote: >> Bump. Can someone from dovecot dev team please respond something to my >> problem ? I've provided all the possible information, and if not please >> tell me. >> Or do you only provide support for your own FTS solution bundled with >> the paid version of dovecot-pro ? If so, you should clearly state it in >> the wiki. >> >> Thanks >> >> Le 19/04/2018 à 10:30, kada...@gmail.com a écrit : >>> I think I'm progressing: if I create a brand new user, send her an >>> email, it will not be indexed right away. I need to doveadm rescan / >>> index the Inbox before. >>> At the end of this email, a dovecot's debug log snipset (grep'ed on this >>> user). In it I can read: "Debug: fts: Indexes disabled for namespace '' >>> Can this be a problem ? >>> >>> Thanks >>> >>> doveconf -n output: >>> # 2.2.34 (874deae): /usr/local/etc/dovecot/dovecot.conf >>> # Pigeonhole version 0.4.22 (22940fb7) >>> # OS: Linux 3.10.0-693.17.1.el7.x86_64 x86_64 CentOS Linux release >>> 7.4.1708 (Core) ext4 >>> # Hostname: mailServer.myDomain >>> auth_master_user_separator = * >>> auth_socket_path = /var/run/dovecot/auth-userdb >>> auth_username_chars = >>> abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@* >>> auth_verbose = yes >>> base_dir = /var/run/dovecot/ >>> debug_log_path = /var/log/dovecot-debug.log >>> default_client_limit = 5120 >>> default_process_limit = 1024 >>> default_vsz_limit = 2 G >>> hostname = myDomain >>> info_log_path = /var/log/dovecot.log >>> lda_mailbox_autocreate = yes >>> lda_mailbox_autosubscribe = yes >>> listen = * >>> log_path = /var/log/dovecot-error.log >>> login_greeting = Welcome >>> login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c >>> mail_gid = vmail >>> mail_location = maildir:/var/vmail/%Ln >>> mail_plugins = " quota stats fts fts_solr" >>> mail_uid = vmail >>> managesieve_notify_capability = mailto >>> managesieve_sieve_capability = fileinto reject envelope >>> encoded-character vacation subaddress comparator-i;ascii-numeric >>> relational regex imap4flags copy include variables body enotify >>> environment mailbox date index ihave duplicate mime foreverypart extracttext >>> namespace { >>> inbox = yes >>> location = maildir:/var/vmail/%Ln >>> mailbox Drafts { >>> auto = subscribe >>> special_use = \Drafts >>> } >>> mailbox Sent { >>> auto = subscribe >>> special_use = \Sent >>> } >>> mailbox "Sent Messages" { >>> special_use = \Sent >>> } >>> mailbox Trash { >>> auto = subscribe >>> special_use = \Trash >>> } >>> prefix = >>> type = private >>> } >>> passdb { >>> args = /usr/local/etc/dovecot/deny-users >>> deny = yes >>> driver = passwd-file >>> } >>> passdb { >>> args = /usr/local/etc/dovecot/master-users >>> driver = passwd-file >>> master = yes >>> } >>> passdb { >>> args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext >>> driver = ldap >>> } >>> plugin { >>> fts = solr >>> fts_autoindex = yes >>> fts_solr = url=http://localhost:8983/solr/dovecot/ break-imap-search >>> quota = maildir:User quota >>> quota_rule = *:storage=20GB >>> quota_rule2 = Trash:storage=+200M >>> quota_warning = storage=95%% quota-warning 95 %u >>> quota_warning2 = storage=80%% quota-warning 80 %u >>> sieve = /var/sieve-scripts/%Ln.sieve >>> sieve_dir = /var/vmail/%Ln/home/sieve >>> stats_command_min_time = 1 mins >>> stats_domain_min_time = 12 hours >>> stats_ip_min_time = 12 hours >>> stats_memory_limit = 16 M >>> stats_refresh = 30 secs >>> stats_session_min_time = 15 mins >>> stats_track_cmds = yes >>> stats_user_min_time = 1 hours >>> } >>> postmaster_address = postmaster@myDomain >>> protocols = imap pop3 sieve sieve >>> service auth-worker { >>> user = root >>> } >>> service auth { >>> unix_listener /var/spool/postfix/private/auth { >>> mode = 0777 >>> user = vmail >>> } >>> } >>> service imap-login { >>> inet_listener imaps { >>> port = 993 >>> ssl = yes >>> } >>> process_min_avail = 10 >>> } >>> service imap { >>> process_limit = 2128 >>> } >>> service managesieve-login { >>> inet_listener sieve { >>> port = 4190 >>> } >>> } >>> service pop3-login { >>> inet_listener pop3s { >>> port = 995
Re: Bug: Dovecot index loosing sync with FTS despite "fts_autoindex = yes"
Bump. Can someone from dovecot dev team please respond something to my problem ? I've provided all the possible information, and if not please tell me. Or do you only provide support for your own FTS solution bundled with the paid version of dovecot-pro ? If so, you should clearly state it in the wiki. Thanks Le 19/04/2018 à 10:30, kada...@gmail.com a écrit : > I think I'm progressing: if I create a brand new user, send her an > email, it will not be indexed right away. I need to doveadm rescan / > index the Inbox before. > At the end of this email, a dovecot's debug log snipset (grep'ed on this > user). In it I can read: "Debug: fts: Indexes disabled for namespace '' > Can this be a problem ? > > Thanks > > doveconf -n output: > # 2.2.34 (874deae): /usr/local/etc/dovecot/dovecot.conf > # Pigeonhole version 0.4.22 (22940fb7) > # OS: Linux 3.10.0-693.17.1.el7.x86_64 x86_64 CentOS Linux release > 7.4.1708 (Core) ext4 > # Hostname: mailServer.myDomain > auth_master_user_separator = * > auth_socket_path = /var/run/dovecot/auth-userdb > auth_username_chars = > abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@* > auth_verbose = yes > base_dir = /var/run/dovecot/ > debug_log_path = /var/log/dovecot-debug.log > default_client_limit = 5120 > default_process_limit = 1024 > default_vsz_limit = 2 G > hostname = myDomain > info_log_path = /var/log/dovecot.log > lda_mailbox_autocreate = yes > lda_mailbox_autosubscribe = yes > listen = * > log_path = /var/log/dovecot-error.log > login_greeting = Welcome > login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c > mail_gid = vmail > mail_location = maildir:/var/vmail/%Ln > mail_plugins = " quota stats fts fts_solr" > mail_uid = vmail > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric > relational regex imap4flags copy include variables body enotify > environment mailbox date index ihave duplicate mime foreverypart extracttext > namespace { > inbox = yes > location = maildir:/var/vmail/%Ln > mailbox Drafts { > auto = subscribe > special_use = \Drafts > } > mailbox Sent { > auto = subscribe > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > mailbox Trash { > auto = subscribe > special_use = \Trash > } > prefix = > type = private > } > passdb { > args = /usr/local/etc/dovecot/deny-users > deny = yes > driver = passwd-file > } > passdb { > args = /usr/local/etc/dovecot/master-users > driver = passwd-file > master = yes > } > passdb { > args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext > driver = ldap > } > plugin { > fts = solr > fts_autoindex = yes > fts_solr = url=http://localhost:8983/solr/dovecot/ break-imap-search > quota = maildir:User quota > quota_rule = *:storage=20GB > quota_rule2 = Trash:storage=+200M > quota_warning = storage=95%% quota-warning 95 %u > quota_warning2 = storage=80%% quota-warning 80 %u > sieve = /var/sieve-scripts/%Ln.sieve > sieve_dir = /var/vmail/%Ln/home/sieve > stats_command_min_time = 1 mins > stats_domain_min_time = 12 hours > stats_ip_min_time = 12 hours > stats_memory_limit = 16 M > stats_refresh = 30 secs > stats_session_min_time = 15 mins > stats_track_cmds = yes > stats_user_min_time = 1 hours > } > postmaster_address = postmaster@myDomain > protocols = imap pop3 sieve sieve > service auth-worker { > user = root > } > service auth { > unix_listener /var/spool/postfix/private/auth { > mode = 0777 > user = vmail > } > } > service imap-login { > inet_listener imaps { > port = 993 > ssl = yes > } > process_min_avail = 10 > } > service imap { > process_limit = 2128 > } > service managesieve-login { > inet_listener sieve { > port = 4190 > } > } > service pop3-login { > inet_listener pop3s { > port = 995 > ssl = yes > } > } > service quota-warning { > executable = script /usr/local/bin/quota-warning.sh > user = nobody > } > service stats { > fifo_listener stats-mail { > mode = 0600 > user = vmail > } > } > ssl_cert = ssl_key = # hidden, use -P to show it > userdb { > args = /usr/local/etc/dovecot/dovecot-ldap-userdb.conf.ext > driver = ldap > override_fields = uid=vmail gid=vmail home=/var/vmail/%Ln/home > } > verbose_proctitle = yes > protocol lda { > mail_plugins = " quota stats fts fts_solr sieve virtual" > } > protocol imap { > mail_max_userip_connections = 20 > mail_plugins = " quota stats fts fts_solr imap_quota virtual imap_stats" > } > protocol sieve { > mail_max_userip_connections = 20 > } > > DEBUG LOG: > Apr 19 10:23:26 lda(NewUser@myDomain): Debug: Loading modules from > directory: /usr/local/lib/dovecot > Apr 19 10:23:26 lda(NewUser@myDomain): Debug: Module loaded: > /usr/local/lib/dovecot/lib10_quota_plugin.so > Apr 19 10:23:26 lda(NewUser@myDomain): Debug: Module l
Re: Bug: Dovecot index loosing sync with FTS despite "fts_autoindex = yes"
I think I'm progressing: if I create a brand new user, send her an email, it will not be indexed right away. I need to doveadm rescan / index the Inbox before. At the end of this email, a dovecot's debug log snipset (grep'ed on this user). In it I can read: "Debug: fts: Indexes disabled for namespace '' Can this be a problem ? Thanks doveconf -n output: # 2.2.34 (874deae): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.22 (22940fb7) # OS: Linux 3.10.0-693.17.1.el7.x86_64 x86_64 CentOS Linux release 7.4.1708 (Core) ext4 # Hostname: mailServer.myDomain auth_master_user_separator = * auth_socket_path = /var/run/dovecot/auth-userdb auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@* auth_verbose = yes base_dir = /var/run/dovecot/ debug_log_path = /var/log/dovecot-debug.log default_client_limit = 5120 default_process_limit = 1024 default_vsz_limit = 2 G hostname = myDomain info_log_path = /var/log/dovecot.log lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = * log_path = /var/log/dovecot-error.log login_greeting = Welcome login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c mail_gid = vmail mail_location = maildir:/var/vmail/%Ln mail_plugins = " quota stats fts fts_solr" mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace { inbox = yes location = maildir:/var/vmail/%Ln mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = type = private } passdb { args = /usr/local/etc/dovecot/deny-users deny = yes driver = passwd-file } passdb { args = /usr/local/etc/dovecot/master-users driver = passwd-file master = yes } passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { fts = solr fts_autoindex = yes fts_solr = url=http://localhost:8983/solr/dovecot/ break-imap-search quota = maildir:User quota quota_rule = *:storage=20GB quota_rule2 = Trash:storage=+200M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = /var/sieve-scripts/%Ln.sieve sieve_dir = /var/vmail/%Ln/home/sieve stats_command_min_time = 1 mins stats_domain_min_time = 12 hours stats_ip_min_time = 12 hours stats_memory_limit = 16 M stats_refresh = 30 secs stats_session_min_time = 15 mins stats_track_cmds = yes stats_user_min_time = 1 hours } postmaster_address = postmaster@myDomain protocols = imap pop3 sieve sieve service auth-worker { user = root } service auth { unix_listener /var/spool/postfix/private/auth { mode = 0777 user = vmail } } service imap-login { inet_listener imaps { port = 993 ssl = yes } process_min_avail = 10 } service imap { process_limit = 2128 } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3s { port = 995 ssl = yes } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh user = nobody } service stats { fifo_listener stats-mail { mode = 0600 user = vmail } } ssl_cert =
Re: Bug: Dovecot index loosing sync with FTS despite "fts_autoindex = yes"
Le 17/04/2018 à 14:18, kfx a écrit : > dovecot 2.2.34 > solr 7.2 > > I only see new messages after typing on the server "doveadm fts rescan > -u username" though I've followed the wiki and added "fts_autoindex = > yes" in 90-plugin.conf . Subsequent search for the same pattern always > gives the same result, ignoring new emails with that particular pattern. > Solr is working correctly and the correct search results comes > immediately (whatever the searched word) after a "doveadm fts rescan" > > # doveconf | grep fts > mail_plugins = " quota stats fts fts_solr" > fts = solr > fts_autoindex = yes > fts_solr = url=http://localhost:8983/solr/dovecot/ > [POP] mail_plugins = " quota stats fts fts_solr sieve virtual" > [IMAP] mail_plugins = " quota stats fts fts_solr imap_quota virtual > imap_stats" > > > > Here are some debug logs from a search (responding with incorrect - or old - results, as new emails have arrived, containing the same "MySearchPattern" pattern). This leads me to a problem on the dovecot side. Only a "doveadm fts rescan -u username" can solve this situation, so should I add a cronjob for it ? It defeats the purpose of solr as the first search after is really slow (but correct). Here are the logs, please tell me if I should provide more info, and thanks for any help: Apr 17 21:44:56 imap(MyUsername): Debug: http-client: host localhost: Host created Apr 17 21:44:56 imap(MyUsername): Debug: http-client: host localhost: DNS lookup successful; got 2 IPs Apr 17 21:44:56 imap(MyUsername): Debug: http-client: peer [::1]:8983: Peer created Apr 17 21:44:56 imap(MyUsername): Debug: http-client: queue http://localhost:8983: Setting up connection to [::1]:8983 (1 requests pending) Apr 17 21:44:56 imap(MyUsername): Debug: http-client: peer [::1]:8983: Linked queue http://localhost:8983 (1 queues linked) Apr 17 21:44:56 imap(MyUsername): Debug: http-client: queue http://localhost:8983: Started new connection to [::1]:8983 Apr 17 21:44:56 imap(MyUsername): Debug: http-client: request [Req1: GET http://localhost:8983/solr/dovecot/select?fl=uid,score&rows=25495&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7d(hdr:MySearchPattern+OR+body:MySearchPattern)&fq=%2Bbox:e09cce0283e8695ab76002deed92+%2Buser:MyUsername]: Submitted Apr 17 21:44:56 imap(MyUsername): Debug: http-client: Waiting for 1 requests to finish Apr 17 21:44:56 imap(MyUsername): Debug: http-client: peer [::1]:8983: Creating 1 new connections to handle requests (already 0 usable, connecting to 0, closing 0) Apr 17 21:44:56 imap(MyUsername): Debug: http-client: peer [::1]:8983: Making new connection 1 of 1 Apr 17 21:44:56 imap(MyUsername): Debug: http-client: conn [::1]:8983 [0]: HTTP connection created (1 parallel connections exist) Apr 17 21:44:56 imap(MyUsername): Debug: http-client: conn [::1]:8983 [0]: Connected Apr 17 21:44:56 imap(MyUsername): Debug: http-client: conn [::1]:8983 [0]: Ready for requests Apr 17 21:44:56 imap(MyUsername): Debug: http-client: peer [::1]:8983: Successfully connected (connections=1) Apr 17 21:44:56 imap(MyUsername): Debug: http-client: peer [::1]:8983: Using 1 idle connections to handle 1 requests (1 total connections ready) Apr 17 21:44:56 imap(MyUsername): Debug: http-client: queue http://localhost:8983: Connection to peer [::1]:8983 claimed request [Req1: GET http://localhost:8983/solr/dovecot/select?fl=uid,score&rows=25495&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7d(hdr:MySearchPattern+OR+body:MySearchPattern)&fq=%2Bbox:e09cce0283e8695ab76002deed92+%2Buser:MyUsername] Apr 17 21:44:56 imap(MyUsername): Debug: http-client: conn [::1]:8983 [0]: Claimed request [Req1: GET http://localhost:8983/solr/dovecot/select?fl=uid,score&rows=25495&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7d(hdr:MySearchPattern+OR+body:MySearchPattern)&fq=%2Bbox:e09cce0283e8695ab76002deed92+%2Buser:MyUsername] Apr 17 21:44:56 imap(MyUsername): Debug: http-client: request [Req1: GET http://localhost:8983/solr/dovecot/select?fl=uid,score&rows=25495&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7d(hdr:MySearchPattern+OR+body:MySearchPattern)&fq=%2Bbox:e09cce0283e8695ab76002deed92+%2Buser:MyUsername]: Sent header Apr 17 21:44:56 imap(MyUsername): Debug: http-client: peer [::1]:8983: No more requests to service for this peer (1 connections exist) Apr 17 21:44:56 imap(MyUsername): Debug: http-client: conn [::1]:8983 [0]: Got 200 response for request [Req1: GET http://localhost:8983/solr/dovecot/select?fl=uid,score&rows=25495&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7d(hdr:MySearchPattern+OR+body:MySearchPattern)&fq=%2Bbox:e09cce0283e8695ab76002deed92+%2Buser:MyUsername] (took 109 ms + 23 ms in queue) Apr 17 21:44:56 imap(MyUsername): Debug: http-client: conn [::1]:8983 [0]: Response payload stream destroyed (0 ms after initial response) Apr 17 21:44:56 imap(MyUsername): Debug: http-client: request [Req1: GET http://localhost:8983/solr/dovecot/select?fl=uid,score&rows=25495&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7d(hdr:MySearchPattern+OR+body:MySearchP
[Dovecot] stats: Error: Mail server input error: UPDATE-SESSION username imap: stats shrank: mrbytes 26961885 < 28237323
Hi List, If anyone is still reading this list, I have another problem beside the FTS's one. From time to time I have this showing up in dovecot-error.log (for several users, not always the same one): stats: Error: Mail server input error: UPDATE-SESSION username imap: stats shrank: mrbytes 26961885 < 28237323 A search in google tell me that it should have been corrected in 2.1.13 but I am using 2.2.12. Is there a human here that can direct me to a solution ? Thanks # 2.2.12: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-431.11.2.el6.x86_64 x86_64 CentOS release 6.5 (Final) ext4 auth_cache_size = 10 M auth_debug = yes auth_master_user_separator = * auth_socket_path = /var/run/dovecot/auth-userdb auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@* auth_verbose = yes auth_verbose_passwords = sha1 base_dir = /var/run/dovecot/ debug_log_path = /var/log/dovecot-debug.log default_client_limit = 2128 default_process_limit = 512 default_vsz_limit = 2 G hostname = mydomain.com info_log_path = /var/log/dovecot.log lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = * log_path = /var/log/dovecot-error.log login_greeting = Welcome login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c mail_debug = yes mail_gid = vmail mail_location = maildir:/var/vmail/%Ln mail_plugins = fts fts_lucene quota stats mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { inbox = yes location = maildir:/var/vmail/%Ln mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = type = private } passdb { args = /usr/local/etc/dovecot/deny-users deny = yes driver = passwd-file } passdb { args = /usr/local/etc/dovecot/master-users driver = passwd-file master = yes } passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { fts = lucene fts_autoindex = yes fts_autoindex_max_recent = 1000 fts_lucene = whitespace_chars=@ quota = maildir:User quota quota_rule = *:storage=20GB quota_rule2 = Trash:storage=+200M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = /var/sieve-scripts/%Ln.sieve sieve_dir = /var/vmail/%Ln/home/sieve stats_command_min_time = 1 mins stats_domain_min_time = 12 hours stats_ip_min_time = 12 hours stats_memory_limit = 16 M stats_refresh = 30 secs stats_session_min_time = 15 mins stats_track_cmds = yes stats_user_min_time = 1 hours } postmaster_address = postmas...@mydomain.com protocols = imap pop3 sieve sieve service auth-worker { user = root } service auth { unix_listener /var/spool/postfix/private/auth { mode = 0777 user = vmail } } service decode2text { executable = script /usr/local/libexec/dovecot/decode2text.sh unix_listener decode2text { mode = 0666 } user = dovecot } service imap-login { inet_listener imaps { port = 993 ssl = yes } process_min_avail = 5 } service imap { process_limit = 1024 } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3s { port = 995 ssl = yes } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh user = nobody } service stats { fifo_listener stats-mail { mode = 0600 user = vmail } } ssl_cert =
[Dovecot] Migration to Dovecot 2.2.12 - How to trigger full site indexing
Hi list, We are in the process of migrating our old (dovecot 1.2.4 based) mail system to the new one: - Centos 6 x86_64 - Dovecot 2.2.12 - Users in an openldap directory - clucene FTS I'm trying to trigger a full indexing for all the user and all their mailboxes (avoiding them to individually trigger it using a search within their MUA). I'm trying to use the "doveadm index" command. First question, is there some sort of 'wildcard' option for the name of the mailbox to index, as I dont know all the mailboxes's names the users have created (beside the regular Sent Inbox Trash and Drafts ? Second question I've tried: # doveadm -D index -A Sent which gave me: doveadm(root): Error: User listing returned failure doveadm: Error: Failed to iterate through some users Can the '-A' option work with ldap backend ? Thanks a lot and have a nice day, kfx - # 2.2.12: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-431.11.2.el6.x86_64 x86_64 CentOS release 6.5 (Final) ext4 auth_cache_size = 10 M auth_debug = yes auth_socket_path = /var/run/dovecot/auth-userdb auth_verbose = yes auth_verbose_passwords = sha1 base_dir = /var/run/dovecot/ debug_log_path = /var/log/dovecot-debug.log default_vsz_limit = 2 G hostname = mydomain.com info_log_path = /var/log/dovecot.log lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = * log_path = /var/log/dovecot-error.log login_greeting = Welcome login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c mail_debug = yes mail_gid = vmail mail_location = maildir:/var/vmail/%Ln mail_plugins = fts fts_lucene quota stats mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { inbox = yes location = maildir:/var/vmail/%Ln mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = type = private } passdb { args = /usr/local/etc/dovecot/deny-users deny = yes driver = passwd-file } passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { fts = lucene fts_autoindex = yes fts_autoindex_max_recent = 1000 fts_lucene = whitespace_chars=@ quota = maildir:User quota quota_rule = *:storage=20GB quota_rule2 = Trash:storage=+200M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = /var/sieve-scripts/%Ln.sieve sieve_dir = /var/vmail/%Ln/home/sieve stats_command_min_time = 1 mins stats_domain_min_time = 12 hours stats_ip_min_time = 12 hours stats_memory_limit = 16 M stats_refresh = 30 secs stats_session_min_time = 15 mins stats_track_cmds = yes stats_user_min_time = 1 hours } postmaster_address = postmas...@mydomain.com protocols = imap pop3 sieve sieve service auth-worker { user = root } service auth { unix_listener /var/spool/postfix/private/auth { mode = 0777 user = vmail } } service imap-login { inet_listener imaps { port = 993 ssl = yes } process_min_avail = 5 } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3s { port = 995 ssl = yes } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh user = nobody } service stats { fifo_listener stats-mail { mode = 0600 user = vmail } } ssl_cert =
Re: [Dovecot] Crash while moving mail between private folder and shared folder
Le 31/03/13 16:26, Timo Sirainen a écrit : On 27.3.2013, at 18.01, kada...@gmail.com wrote: Just to bump the OP message as I experience the exact same symptom with the same configuration. That is: when moving an email to a virtual mailboxes, dovecot crash (and sometime the email disappears from the real mailbox). I'm guessing your error was different though. These should fix them: http://hg.dovecot.org/dovecot-2.2/rev/12136db6e31f http://hg.dovecot.org/dovecot-2.2/rev/9a8119326bd0 (not sure if http://hg.dovecot.org/dovecot-2.2/rev/f31d241d5371 is also required) Thanks Timo, this fixed my issue (yesterday's pull of the mercurial repo). You are the internet IMAP Hero. Have a nice day, kfx
Re: [Dovecot] Crash while moving mail between private folder and shared folder
Hi, Just to bump the OP message as I experience the exact same symptom with the same configuration. That is: when moving an email to a virtual mailboxes, dovecot crash (and sometime the email disappears from the real mailbox). Thx Le 23/03/13 10:03, Christian Felsing a écrit : Hello, while moving a mail from a private folder (inbox) to a shared folder and back to private folder Dovecot 2.2rc3 crashes: ---cut here--- Mar 23 09:57:44 dovecot dovecot: imap-login: Login: user=, method=PLAIN, rip=192.168.200.6, lip=192.168.200.22, mpid=19423, TLS Mar 23 09:57:44 dovecot dovecot: imap(john@example.net): Panic: file mail-namespace.c: line 654 (mail_namespace_find): assertion failed: (ns != NULL) Mar 23 09:57:44 dovecot dovecot: imap(john@example.net): Error: Raw backtrace: /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot.so.0(+0x608fa) [0x7f82f65b28fa] -> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot.so.0(+0x6093e) [0x7f82f65b293e] -> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f82f657435b] -> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot-storage.so.0(mail_namespace_find_unalias+0) [0x7f82f686b590] -> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot-storage.so.0(mail_namespace_find_unalias+0x19) [0x7f82f686b5a9] -> /opt/dovecot-2.2.rc3/lib/dovecot/lib10_quota_plugin.so(+0x57bd) [0x7f82f51127bd] -> /opt/dovecot-2.2.rc3/lib/dovecot/lib10_quota_plugin.so(quota_test_alloc+0xcb) [0x7f82f511424b] -> /opt/dovecot-2.2.rc3/lib/dovecot/lib10_quota_plugin.so(quota_try_alloc+0x3d) [0x7f82f511445d] -> /opt/dovecot-2.2.rc3/lib/dovecot/lib10_quota_plugin.so(+0xb00b) [0x7f82f511800b] -> /opt/dovecot-2.2.rc3/lib/dovecot/lib01_acl_plugin.so(+0xc7cb) [0x7f82f552d7cb] -> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot-storage.so.0(mailbox_copy+0x5f) [0x7f82f687274f] -> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot-storage.so.0(mailbox_move+0x10) [0x7f82f6872810] -> dovecot-01/imap() [0x40d29d] -> dovecot-01/imap(command_exec+0x3c) [0x4161ec] -> dovecot-01/imap() [0x415250] -> dovecot-01/imap() [0x41530a] -> dovecot-01/imap(client_handle_input+0x115) [0x4155c5] -> dovecot-01/imap(client_input+0x72) [0x415972] -> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f82f65c20e6] -> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xd7) [0x7f82f65c2f37] -> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f82f65c1c28] -> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f82f6579473] -> dovecot-01/imap(main+0x270) [0x40b950] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f82f61e6ead] -> dovecot-01/imap() [0x40baad] Mar 23 09:57:44 dovecot dovecot: imap(john@example.net): Fatal: master: service(imap): child 19423 killed with signal 6 (core dumps disabled) Mar 23 09:57:44 dovecot dovecot: imap-login: Login: user=, method=PLAIN, rip=192.168.200.6, lip=192.168.200.22, mpid=19425, TLS Mar 23 09:57:44 dovecot dovecot: imap(john@example.net): Panic: file mail-namespace.c: line 654 (mail_namespace_find): assertion failed: (ns != NULL) Mar 23 09:57:44 dovecot dovecot: imap(john@example.net): Error: Raw backtrace: /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot.so.0(+0x608fa) [0x7f66ed9a58fa] -> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot.so.0(+0x6093e) [0x7f66ed9a593e] -> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f66ed96735b] -> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot-storage.so.0(mail_namespace_find_unalias+0) [0x7f66edc5e590] -> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot-storage.so.0(mail_namespace_find_unalias+0x19) [0x7f66edc5e5a9] -> /opt/dovecot-2.2.rc3/lib/dovecot/lib10_quota_plugin.so(+0x57bd) [0x7f66ec5057bd] -> /opt/dovecot-2.2.rc3/lib/dovecot/lib10_quota_plugin.so(quota_test_alloc+0xcb) [0x7f66ec50724b] -> /opt/dovecot-2.2.rc3/lib/dovecot/lib10_quota_plugin.so(quota_try_alloc+0x3d) [0x7f66ec50745d] -> /opt/dovecot-2.2.rc3/lib/dovecot/lib10_quota_plugin.so(+0xb00b) [0x7f66ec50b00b] -> /opt/dovecot-2.2.rc3/lib/dovecot/lib01_acl_plugin.so(+0xc7cb) [0x7f66ec9207cb] -> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot-storage.so.0(mailbox_copy+0x5f) [0x7f66edc6574f] -> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot-storage.so.0(mailbox_move+0x10) [0x7f66edc65810] -> dovecot-01/imap() [0x40d29d] -> dovecot-01/imap(command_exec+0x3c) [0x4161ec] -> dovecot-01/imap() [0x415250] -> dovecot-01/imap() [0x41530a] -> dovecot-01/imap(client_handle_input+0x115) [0x4155c5] -> dovecot-01/imap(client_input+0x72) [0x415972] -> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f66ed9b50e6] -> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xd7) [0x7f66ed9b5f37] -> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f66ed9b4c28] -> /opt/dovecot-2.2.rc3/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f66ed96c473] -> dovecot-01/imap(main+0x270) [0x40b950] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f66ed5d9ead] -> d