Re: Moving old emails to Gmail
Wouldn't doveadm user myaddr...@domain.com tell you the mail directory? On 06/11/2021 11.10, Rob Campbell wrote: I have a backup of mail messages I have from back when i was in school using pine and I want to have them imported to my Gmail. So I set up dovecot and my Gmail can access it just fine but it can't find anything except my inbox (which I can't even find). I've moved my mail directory and I can view the emails using alpine but I cannot view my inbox using alpine. Also, although Gmail can get my new test messages, it can't find the messages in ~/mail where I am viewing them with alpine. How do I make it so Gmail can pull from ~/mail or ~/Maildir because I also converted the messages in ~/mail to ~/Maildir format but although I've tried mail_location = mbox:~/mail mail_location = mbox:~/Maildir mail_location = maildir:~/mail and mail_location = maildir:~/Maildir None of this worked but mail_location = maildir:~/Maildir is the only one that will at least allow me to get test messages in my inbox which Gmail can then pull. How do I resolve these issues? ~ In all things, Be Intentional.
Moving old emails to Gmail
I have a backup of mail messages I have from back when i was in school using pine and I want to have them imported to my Gmail. So I set up dovecot and my Gmail can access it just fine but it can't find anything except my inbox (which I can't even find). I've moved my mail directory and I can view the emails using alpine but I cannot view my inbox using alpine. Also, although Gmail can get my new test messages, it can't find the messages in ~/mail where I am viewing them with alpine. How do I make it so Gmail can pull from ~/mail or ~/Maildir because I also converted the messages in ~/mail to ~/Maildir format but although I've tried mail_location = mbox:~/mail mail_location = mbox:~/Maildir mail_location = maildir:~/mail and mail_location = maildir:~/Maildir None of this worked but mail_location = maildir:~/Maildir is the only one that will at least allow me to get test messages in my inbox which Gmail can then pull. How do I resolve these issues? ~ In all things, Be Intentional.
Unable to find mail
I just installed dovecot and it seems to be working well except I can't find where my new mail is being stored. ~ In all things, Be Intentional.
Re: stale (?) .dovecot.svbin causing segfault in dovecot-lda
On 03/11/2021 11:34, Matthieu Herrb wrote:
Hi,
I've not touched the sieve filters I'm using for a long time (last
modification 2 years ago), but I've upgraded the dovecot package and
the system of my mail server.
~/.dovecot.svbin has not been updated, but I found out today that it
would cause dovecot-lda to crash on some specifig messages (and fail
to deliver them). Most of the mails (>99.9%) are delivered ok though.
Here's the trace of the crash in the system logs :
Nov 3 08:48:13 nowhere dovecot: lda(matthieu)<33178>:
Panic: Buffer write out of range (0 + 1)
Nov 3 08:48:15 nowhere smtpd[37379]: ac2aaecba4099baa mda delivery evpid=b9346ef1d2a6c223
from= to= rcpt= user=matthieu delay=12s
result=PermFail stat=Error ("Abort trap (core dumped) ")
After removing the old file, dovecot-lda is able to deliver the
message that caused the crash whitout issues.
Shouldn't ~/.dovecot.svbin be automatically be re-generated on dovecot
version changes ?
Yes, definitely.
Some details:
I'm running OpenBSD (7.0 right now, was 6.9 before and probably 6.5
the last time .dovecot.svbin was generated). My logs show that the
issue has also been happening with OpenBSD 6.9, but I never noticed
until today). OpenSMTP is configured to deliver the message through
dovecot-lda with:
action "deliver" \
mda "/usr/local/libexec/dovecot/dovecot-lda" \
alias
in /etc/mail/smtpd.conf
I'd need at least the Sieve script and the .svbin or, better yet, a
backtrace of the panic core dump.
Regards,
Stephan.
Below is the output of doveadm config :
# 2.3.16 (7e2e900c1a): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.16 (09c29328)
# OS: OpenBSD 7.0 amd64 ffs
# Hostname: nowhere.herrb.eu
# NOTE: Send doveconf -n output instead when asking for help.
auth_anonymous_username = anonymous
auth_cache_negative_ttl = 1 hours
auth_cache_size = 0
auth_cache_ttl = 1 hours
auth_cache_verify_password_with_worker = no
auth_debug = no
auth_debug_passwords = no
auth_default_realm =
auth_failure_delay = 2 secs
auth_gssapi_hostname =
auth_krb5_keytab =
auth_master_user_separator =
auth_mechanisms = plain
auth_policy_check_after_auth = yes
auth_policy_check_before_auth = yes
auth_policy_hash_mech = sha256
auth_policy_hash_nonce =
auth_policy_hash_truncate = 12
auth_policy_log_only = no
auth_policy_reject_on_fail = no
auth_policy_report_after_auth = yes
auth_policy_request_attributes = login=%{requested_username}
pwhash=%{hashed_password} remote=%{rip} device_id=%{client_id} protocol=%s
session_id=%{session}
auth_policy_server_api_header =
auth_policy_server_timeout_msecs = 2000
auth_policy_server_url =
auth_proxy_self =
auth_realms =
auth_socket_path = auth-userdb
auth_ssl_require_client_cert = no
auth_ssl_username_from_cert = no
auth_stats = no
auth_use_winbind = no
auth_username_chars =
abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@
auth_username_format = %Lu
auth_username_translation =
auth_verbose = yes
auth_verbose_passwords = no
auth_winbind_helper_path = /usr/bin/ntlm_auth
auth_worker_max_count = 30
base_dir = /var/dovecot
config_cache_size = 1 M
debug_log_path =
default_client_limit = 500
default_idle_kill = 1 mins
default_internal_group = _dovecot
default_internal_user = _dovecot
default_login_user = _dovenull
default_process_limit = 100
default_vsz_limit = 256 M
deliver_log_format = msgid=%m: %$
dict_db_config =
director_flush_socket =
director_mail_servers =
director_max_parallel_kicks = 100
director_max_parallel_moves = 100
director_output_buffer_size = 10 M
director_ping_idle_timeout = 30 secs
director_ping_max_timeout = 1 mins
director_servers =
director_user_expire = 15 mins
director_user_kick_delay = 2 secs
director_username_hash = %u
disable_plaintext_auth = yes
dotlock_use_excl = yes
doveadm_allowed_commands =
doveadm_api_key =
doveadm_http_rawlog_dir =
doveadm_password =
doveadm_port = 0
doveadm_socket_path = doveadm-server
doveadm_ssl = no
doveadm_username = doveadm
doveadm_worker_count = 0
dsync_alt_char = _
dsync_commit_msgs_interval = 100
dsync_features =
dsync_hashed_headers = Date Message-ID
dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u -U
first_valid_gid = 1
first_valid_uid = 1000
haproxy_timeout = 3 secs
haproxy_trusted_networks =
hostname =
imap_capability =
imap_client_workarounds =
imap_fetch_failure = disconnect-immediately
imap_hibernate_timeout = 0
imap_id_log =
imap_id_retain = no
imap_id_send = name *
imap_idle_notify_interval = 2 mins
imap_literal_minus = no
imap_logout_format = in=%i out=%o deleted=%{deleted} expunged=%{expunged}
trashed=%{trashed} hdr_count=%{fetch_hdr_count} hdr_bytes=%{fetch_hdr_bytes}
body_count=%{fetch_body_count} body_bytes=%{fetch_body_bytes}
imap_max_line_length = 64 k
imap_metadata = no
imap_urlauth_host =
imap_urlauth_logout_format = in=%i out=%o
imap_urlauth_port = 143
imapc_cmd_timeout = 5 mins
imapc_connection_retry_count = 1
imapc_connection_retry_interval = 1 secs
imapc_features =
imapc_host =
imapc
Re: dsync-trouble
Hi, Aki Tuomi schrieb: >Actually, this is caused by fts_autoindex_exclude issue, which has >already been reported several times on the list. I owe You a beer the next time You're in the Frankfurt area. AFAICS things are working again. Regards Hanns
Re: dovecot: 2.3.16 -> 2.3.17_1
Elise: Can you tell how to fix? Thanks! In order to keep SSL alive I downgraded to the pre-last version of Dovcecot and ran the command pkg lock dovecot until an update will be released. ./elise
after replication with compression quotas are wrong
Hello,
We are very long time happy dovecot users (since 2008 at least). We have
around 90k mailboxes and since we had to move away from our NAS storage
to a ceph storage I jumped on the opportunity to enable compression with
the zlib plugin and dovecot's replication mecanism. We are using
debian's dovecot 2.2.27 packages on production and our new server is
running dovecot's own ce-2.3.17 packages.
On the production server everything works fine but on the new server,
replicated mailboxes' quota is all wrong:
on production:
# doveadm quota get -u dummy-c-1
Quota nameType Value Limit %
Quota Utilisateur STORAGE 660026 97656367
Quota Utilisateur MESSAGE 8651 - 0
on new server:
doveadm quota get -u dummy-c-1
Quota nameType Value Limit %
Quota Utilisateur STORAGE 1125251 976563 115
Quota Utilisateur MESSAGE 16646 - 0
If I add all the S flag from the filenames n both servers I get exactly
the same usage, which is coherent with the quota on the production server:
# find . -type f | grep 'S=' | awk -F'S=' '{print $2}' | awk -F','
'{print $1}' | awk -F':' '{print $1}' | paste -sd+ | bc -l
675865938
And I have exactly the same amountof mails on the two server, the
replication works as expected, no unwanted duplication of mails occurs.
Of course, I've tried to ask dovecot to recalculate quotas with doveadm
quota recalc -u , but it doesn't fix the problem.
What am I missing?
Thanks,
Arnaud
PS: Here is my doveconf -n output:
# 2.3.17 (e2aa53df5b): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.17 (054dddfa)
# OS: Linux 4.19.0-18-amd64 x86_64 Debian 10.11
# Hostname: gromel-test
auth_cache_size = 10 k
auth_verbose = yes
disable_plaintext_auth = no
doveadm_password = # hidden, use -P to show it
hostname = gromel1.univ-nantes.prive
lda_mailbox_autosubscribe = yes
listen = *
log_timestamp = "%Y-%m-%d %H:%M:%S "
login_trusted_networks = (...)
mail_gid = 5000
mail_location = maildir:%h
mail_plugins = quota zlib notify replication
mail_privileged_group = vmail
mail_uid = 5000
maildir_stat_dirs = yes
maildir_very_dirty_syncs = yes
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope
encoded-character subaddress comparator-i;ascii-numeric relational regex
imap4flags copy include variables body enotify environment mailbox date
index ihave duplicate mime foreverypart extracttext
namespace {
inbox = yes
location =
prefix = INBOX.
separator = .
type = private
}
passdb {
args = /etc/dovecot/dovecot-ldap.conf
driver = ldap
}
plugin {
quota = maildir:Quota Utilisateur
quota_exceeded_message = Cet utilisateur a dépassé son quota, votre
message n'a pu lui être livré.
quota_full_tempfail = yes
quota_rule = *:storage=1000M
quota_rule2 = INBOX.Trash:storage=+100M
sieve = ~/dovecot.sieve
sieve_dir = ~/sieve
sieve_extensions = -vacation
sieve_global_dir = /var/lib/dovecot/sieve/global/
sieve_max_redirects = 1
zlib_save = gz
zlib_save_level = 6
}
postmaster_address = postmaster@
protocols = imap pop3 sieve
replication_max_conns = 50
service auth {
client_limit = 49452
unix_listener auth-userdb {
group = vmail
mode = 0600
user = vmail
}
user = root
}
service dict {
unix_listener dict {
mode = 0600
user = vmail
}
}
service doveadm {
inet_listener {
port = 12345
}
}
service imap-login {
process_min_avail = 8
service_count = 0
user = mail
}
service imap {
executable = imap
process_limit = 16384
vsz_limit = 2 G
}
service managesieve-login {
inet_listener sieve {
port = 4190
}
process_min_avail = 8
service_count = 0
user = mail
vsz_limit = 2 G
}
service managesieve {
drop_priv_before_exec = yes
process_limit = 16384
}
service pop3-login {
process_min_avail = 8
service_count = 0
user = mail
}
service pop3 {
drop_priv_before_exec = yes
process_limit = 16384
}
ssl = no
userdb {
args = /etc/dovecot/dovecot-ldap.conf
driver = ldap
}
verbose_proctitle = yes
protocol imap {
mail_max_userip_connections = 50
mail_plugins = quota zlib notify replication imap_quota zlib
}
protocol pop3 {
mail_plugins = quota zlib notify replication
}
protocol sieve {
mail_max_userip_connections = 10
}
protocol lda {
mail_plugins = quota zlib notify replication sieve zlib
}
--
Arnaud Abélard
Responsable pôle Système et Stockage
Service Infrastructures
DSIN Université de Nantes
-
Re: Recovering deleted messages?
On 05/11/2021 07.01, Sean McBride wrote: But now I have a user who accidentally used a POP client, and it deleted all his email. He told me this only days later, and it's now been days later still, and so of course he has sent and received email since. I have backups of course, but it is a feasible thing to 'merge' the deleted emails from the backup into his current mailboxes? Personally, I'd restore them to another, temporary account set up for the purpose. Then I'd either use - dovecot sync, - imapsync, or - manual drag and drop from one account to the other in Thunderbird to put the mails back in place, depending on volume of mail involved and personal whim. The Thunderbird route would be easiest for a user to do themselves once set up. However as mail is downloaded from the server to the desktop and then back up to the server, it would take a while if there were lots of messages to deal with. P.
Re: dsync-trouble
Hi, Am 05.11.21 um 09:14 schrieb infoomatic: > On 05.11.21 08:39, Aki Tuomi wrote: >> Not sure how this is done in FreeBSD but can you try getting debug >> symbols somehow? >> >> Aki > > @Hanns: can you compile from ports and use > https://wiki.freebsd.org/BenWoods/DebuggingPorts or just 'make > WITH_DEBUG=yes DEBUG_FLAGS="-g -O0"' in /usr/local/mail/dovecot I've already building dovecot, following Ben's advice. I'm on the road for the rest of the day, so I can't proceed at the Moment. Regards Hanns
Re: dsync-trouble
Hi, Am 05.11.21 um 08:56 schrieb Aki Tuomi: > Actually, this is caused by fts_autoindex_exclude issue, which has already > been reported several times on the list. I can login again, after I've disabled fts. Thank You Still compiling dovecot with debug-symbols. Regards Hanns
Re: dsync-trouble
On 05.11.21 08:39, Aki Tuomi wrote: Not sure how this is done in FreeBSD but can you try getting debug symbols somehow? Aki @Hanns: can you compile from ports and use https://wiki.freebsd.org/BenWoods/DebuggingPorts or just 'make WITH_DEBUG=yes DEBUG_FLAGS="-g -O0"' in /usr/local/mail/dovecot
Re: dsync-trouble
Actually, this is caused by fts_autoindex_exclude issue, which has already been reported several times on the list. Aki > On 05/11/2021 09:39 Aki Tuomi wrote: > > > > On 05/11/2021 09:38 Hanns Mattes wrote: > > > > > > Hi, > > > > Am 04.11.21 um 08:51 schrieb Aki Tuomi: > > > That's in IMAP process, not in doveadm process. > > > > > > Can you try obtaining the core and producing gdb bt full for it? > > > > got coredumps from doveadm-director and imap: > > > > > > gdb /usr/local/libexec/dovecot/imap /var/coredumps/2001.imap.core > > > > Not sure how this is done in FreeBSD but can you try getting debug symbols > somehow? > > Aki
Re: dsync-trouble
> On 05/11/2021 09:38 Hanns Mattes wrote: > > > Hi, > > Am 04.11.21 um 08:51 schrieb Aki Tuomi: > > That's in IMAP process, not in doveadm process. > > > > Can you try obtaining the core and producing gdb bt full for it? > > got coredumps from doveadm-director and imap: > > > gdb /usr/local/libexec/dovecot/imap /var/coredumps/2001.imap.core > Not sure how this is done in FreeBSD but can you try getting debug symbols somehow? Aki
Re: dsync-trouble
Hi, Am 04.11.21 um 08:51 schrieb Aki Tuomi: > That's in IMAP process, not in doveadm process. > > Can you try obtaining the core and producing gdb bt full for it? got coredumps from doveadm-director and imap: gdb /usr/local/libexec/dovecot/imap /var/coredumps/2001.imap.core #0 0x1150b2ea in thr_kill () from /lib/libc.so.7 No symbol table info available. #1 0x11480064 in raise () from /lib/libc.so.7 No symbol table info available. #2 0x11534f29 in abort () from /lib/libc.so.7 No symbol table info available. #3 0x11372d43 in ?? () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #4 0x113714ff in ?? () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #5 0x113722c1 in ?? () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #6 0x1137170b in i_panic () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #7 0x11363274 in array_idx_modifiable_i () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #8 0x12069d98 in fts_user_autoindex_exclude () from /usr/local/lib/dovecot/lib20_fts_plugin.so No symbol table info available. #9 0x1206723b in fts_mailbox_allocated () from /usr/local/lib/dovecot/lib20_fts_plugin.so No symbol table info available. #10 0x11138f8a in hook_mailbox_allocated () from /usr/local/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #11 0x111323a1 in mailbox_alloc () from /usr/local/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #12 0x01043dc9 in cmd_select_full () No symbol table info available. #13 0x0104c6a2 in command_exec () No symbol table info available. #14 0x0104afba in ?? () No symbol table info available. #15 0x0104b0c1 in ?? () No symbol table info available. #16 0x01049ad2 in client_handle_input () No symbol table info available. #17 0x01048228 in client_input () No symbol table info available. #18 0x1138bf2f in io_loop_call_io () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #19 0x1138e964 in io_loop_handler_run_internal () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #20 0x1138c4ca in io_loop_handler_run () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #21 0x1138c308 in io_loop_run () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #22 0x112ef638 in master_service_run () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #23 0x0105a967 in main () No symbol table info available. doveadm-director: #0 0x0008017092ea in thr_kill () from /lib/libc.so.7 No symbol table info available. #1 0x00080167e064 in raise () from /lib/libc.so.7 No symbol table info available. #2 0x000801732f29 in abort () from /lib/libc.so.7 No symbol table info available. #3 0x000801570d43 in ?? () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #4 0x00080156f4ff in ?? () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #5 0x0008015702c1 in ?? () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #6 0x00080156f70b in i_panic () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #7 0x000801561274 in array_idx_modifiable_i () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #8 0x000802831d98 in fts_user_autoindex_exclude () from /usr/local/lib/dovecot/lib20_fts_plugin.so No symbol table info available. #9 0x00080282f23b in fts_mailbox_allocated () from /usr/local/lib/dovecot/lib20_fts_plugin.so No symbol table info available. #10 0x000801336f8a in hook_mailbox_allocated () from /usr/local/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #11 0x0008013303a1 in mailbox_alloc () from /usr/local/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #12 0x01096846 in ?? () No symbol table info available. #13 0x0109609f in dsync_mailbox_tree_fill () No symbol table info available. #14 0x01089fbd in dsync_brain_mailbox_trees_init () No symbol table info available. #15 0x010873ae in dsync_brain_run () No symbol table info available. #16 0x01086873 in ?? () No symbol table info available. #17 0x0109eed4 in ?? () No symbol table info available. #18 0x000801589f2f in io_loop_call_io () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #19 0x00080158c964 in io_loop_handler_run_internal () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #20 0x00080158a4ca in io_loop_handler_run () from /usr/local/lib/dovecot/libdovecot.so.0 No s
