Re: Question about doveadm altmove
Any update on this? Should I fill a bug-report for doveadm altmove -r not working? Regards Javier El 28/03/2021 a las 18:00, JAVIER MIGUEL RODRIGUEZ escribió: Any update on this? Dovecot 2.3.14 makes doveadm altmove -r functional? *De:* dovecot en nombre de María Arrea *Enviado:* Monday, March 22, 2021 3:15:13 PM *Cc:* dovecot@dovecot.org *Asunto:* Re: Question about doveadm altmove zlib plugin, as far as I know, only supports zstd, gzip, bzip2 and lzma/xz compression. The last one is being deprecated. I have found this interesting post in the mailing list: https://dovecot.org/pipermail/dovecot/2021-February/121329.html Same problem here with Dovecot 2.3.13, "doveadm altmove -r" is not moving anything from alternate to default storage. I fixed this by reverting this commit: https://github.com/dovecot/core/commit/2795f6183049a8a4cc489869b3e866dc20a8a732 <https://github.com/dovecot/core/commit/2795f6183049a8a4cc489869b3e866dc20a8a732> Is this fixed in 2.3.14 ? Does doveadm altmove -r works as expected in 2.3.14? Regards *Sent:* Sunday, March 21, 2021 at 11:28 PM *From:* "justina colmena ~biz" *To:* dovecot@dovecot.org *Subject:* Re: Question about doveadm altmove On Sunday, March 21, 2021 12:16:28 PM AKDT María Arrea wrote: > Hello. > > We are running dovecot 2.3.13. Full doveconf -n output below > > In 2.3.14 Changelog I found this: > > * Remove XZ/LZMA write support. Read support will be removed in future > release. > We are using mdbox + XZ/LZMA for alternate storage (messages older than 2 > weeks are moved to ALT storage via cron job), so we must convert from XZ to > another thing (maybe zstd or bz2). Why can't you just pipe the output of "doveadm altmove" command through an external command to do the XZ/LZMA compression if dovecot no longer supports it internally? From doveadm-altmove (1): > This command can be used with sdbox or mdbox storage to move mails to alternative > storage path when :ALT= is specified for the mail location. And that's set in stone. https://en.wikipedia.org/wiki/XZ_Utils <https://en.wikipedia.org/wiki/XZ_Utils> So what are the issues with xz? Security? Crashes or viruses on expanding invalid archives?
Re: Question about doveadm altmove
Any update on this? Dovecot 2.3.14 makes doveadm altmove -r functional? De: dovecot en nombre de María Arrea Enviado: Monday, March 22, 2021 3:15:13 PM Cc: dovecot@dovecot.org Asunto: Re: Question about doveadm altmove zlib plugin, as far as I know, only supports zstd, gzip, bzip2 and lzma/xz compression. The last one is being deprecated. I have found this interesting post in the mailing list: https://dovecot.org/pipermail/dovecot/2021-February/121329.html Same problem here with Dovecot 2.3.13, "doveadm altmove -r" is not moving anything from alternate to default storage. I fixed this by reverting this commit: https://github.com/dovecot/core/commit/2795f6183049a8a4cc489869b3e866dc20a8a732 Is this fixed in 2.3.14 ? Does doveadm altmove -r works as expected in 2.3.14? Regards Sent: Sunday, March 21, 2021 at 11:28 PM From: "justina colmena ~biz" To: dovecot@dovecot.org Subject: Re: Question about doveadm altmove On Sunday, March 21, 2021 12:16:28 PM AKDT María Arrea wrote: > Hello. > > We are running dovecot 2.3.13. Full doveconf -n output below > > In 2.3.14 Changelog I found this: > > * Remove XZ/LZMA write support. Read support will be removed in future > release. > We are using mdbox + XZ/LZMA for alternate storage (messages older than 2 > weeks are moved to ALT storage via cron job), so we must convert from XZ to > another thing (maybe zstd or bz2). Why can't you just pipe the output of "doveadm altmove" command through an external command to do the XZ/LZMA compression if dovecot no longer supports it internally? >From doveadm-altmove (1): > This command can be used with sdbox or mdbox storage to move mails to alternative > storage path when :ALT= is specified for the mail location. And that's set in stone. https://en.wikipedia.org/wiki/XZ_Utils So what are the issues with xz? Security? Crashes or viruses on expanding invalid archives?
Re: Question about doveadm altmove
zlib plugin, as far as I know, only supports zstd, gzip, bzip2 and lzma/xz compression. The last one is being deprecated. I have found this interesting post in the mailing list: https://dovecot.org/pipermail/dovecot/2021-February/121329.html Same problem here with Dovecot 2.3.13, "doveadm altmove -r" is not moving anything from alternate to default storage. I fixed this by reverting this commit: https://github.com/dovecot/core/commit/2795f6183049a8a4cc489869b3e866dc20a8a732 Is this fixed in 2.3.14 ? Does doveadm altmove -r works as expected in 2.3.14? Regards Sent: Sunday, March 21, 2021 at 11:28 PM From: "justina colmena ~biz" To: dovecot@dovecot.org Subject: Re: Question about doveadm altmove On Sunday, March 21, 2021 12:16:28 PM AKDT María Arrea wrote: > Hello. > > We are running dovecot 2.3.13. Full doveconf -n output below > > In 2.3.14 Changelog I found this: > > * Remove XZ/LZMA write support. Read support will be removed in future > release. > We are using mdbox + XZ/LZMA for alternate storage (messages older than 2 > weeks are moved to ALT storage via cron job), so we must convert from XZ to > another thing (maybe zstd or bz2). Why can't you just pipe the output of "doveadm altmove" command through an external command to do the XZ/LZMA compression if dovecot no longer supports it internally? From doveadm-altmove (1): > This command can be used with sdbox or mdbox storage to move mails to alternative > storage path when :ALT= is specified for the mail location. And that's set in stone. https://en.wikipedia.org/wiki/XZ_Utils So what are the issues with xz? Security? Crashes or viruses on expanding invalid archives?
Re: Question about doveadm altmove
On Sunday, March 21, 2021 12:16:28 PM AKDT María Arrea wrote: > Hello. > > We are running dovecot 2.3.13. Full doveconf -n output below > > In 2.3.14 Changelog I found this: > > * Remove XZ/LZMA write support. Read support will be removed in future > release. > We are using mdbox + XZ/LZMA for alternate storage (messages older than 2 > weeks are moved to ALT storage via cron job), so we must convert from XZ to > another thing (maybe zstd or bz2). Why can't you just pipe the output of "doveadm altmove" command through an external command to do the XZ/LZMA compression if dovecot no longer supports it internally? From doveadm-altmove (1): > This command can be used with sdbox or mdbox storage to move mails to alternative > storage path when :ALT= is specified for the mail location. And that's set in stone. https://en.wikipedia.org/wiki/XZ_Utils So what are the issues with xz? Security? Crashes or viruses on expanding invalid archives? signature.asc Description: This is a digitally signed message part.
Question about doveadm altmove
Hello.
We are running dovecot 2.3.13. Full doveconf -n output below
In 2.3.14 Changelog I found this:
* Remove XZ/LZMA write support. Read support will be removed in future release.
We are using mdbox + XZ/LZMA for alternate storage (messages older than 2 weeks are moved to ALT storage via cron job), so we must convert from XZ to another thing (maybe zstd or bz2).
Our cron job script:
/usr/bin/doveadm -o mdbox_rotate_interval=14d -o mdbox_purge_preserve_alt=no -o mdbox_rotate_size=250m -o zlib_save=xz -o zlib_save_level=9 altmove -u $user
My first try to accomplish this conversion, without downtime, is this
* Pick an user
* Move all ALT messages from that user mailbox to MAIN storage location (doveadm altmove -r)
* Use doveadm altmove with zstd to recompress messages to ALT
I have picked an user, and tried this:
/usr/bin/doveadm altmove -r -u myuser all
But it did not work. I expected with that command to move ALL messages from ALT storage to main storage, but 0 messages moved, checked with du -ksh
What is the proper way to recompress messages in ALT storage in mdbox?
doveconf -n output:
auth_cache_negative_ttl = 10 secs
auth_cache_size = 10 M
auth_cache_ttl = 2 mins
auth_master_user_separator = *
auth_worker_max_count = 5500
base_dir = /var/run/dovecot/
default_client_limit = 5000
default_process_limit = 6500
disable_plaintext_auth = no
imap_capability = +XLIST
imap_client_workarounds = tb-extra-mailbox-sep delay-newmail tb-lsub-flags
imap_id_log = *
imap_max_line_length = 92 k
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
mail_fsync = never
mail_gid = entrega
mail_home = /buzones_primarios/%2.26Hn/%2.200Hn/%n/home_usuario
mail_location = mdbox:/buzones_primarios/%2.26Hn/%2.200Hn/%n:ALT=/buzones_secundarios/%2.26Hn/%2.200Hn/%n:INDEX=/indices_correo/%2.26Hn/%2.200Hn/%n:ITERINDEX
mail_log_prefix = "%s(%u)<%{session}>: "
mail_max_userip_connections = 2500
mail_plugins = " zlib acl quota virtual"
mail_uid = entrega
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date
mdbox_rotate_interval = 1 days
mdbox_rotate_size = 60 M
namespace {
inbox = yes
location =
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox SPAM {
auto = subscribe
special_use = \Junk
}
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox Trash {
auto = subscribe
special_use = \Trash
}
prefix =
}
passdb {
args = /etc/dovecot/deny
deny = yes
driver = passwd-file
}
passdb {
args = /etc/dovecot/dovecot-ldap.conf
driver = ldap
}
passdb {
args = /etc/usuario_maestro.txt
driver = passwd-file
master = yes
}
passdb {
args = /etc/dovecot/dovecot-ldap.conf
driver = ldap
}
plugin {
acl = vfile
lda_mailbox_autosubscribe = yes
quota = dict:Cuota de usuario::file:/indices_correo/cuotas/%n
quota_rule2 = Trash:storage=+10%%
quota_warning = storage=90%% aviso_cuota 90 %u
sieve = /buzones_primarios/%2.26Hn/%2.200Hn/%n/home_usuario/dovecot.sieve
sieve_dir = /buzones_primarios/%2.26Hn/%2.200Hn/%n/home_usuario/sieve/
sieve_max_redirects = 20
zlib_save = gz
zlib_save_level = 1
}
pop3_no_flag_updates = yes
protocols = pop3 imap sieve
service anvil {
client_limit = 25000
}
service auth {
client_limit = 32000
unix_listener auth-master {
user = entrega
}
unix_listener auth-userdb {
user = entrega
}
user = root
}
service aviso_cuota {
executable = script /usr/local/bin/quota-warning.sh
unix_listener aviso_cuota {
mode = 0666
}
user = entrega
}
service config {
process_min_avail = 4
}
service imap-login {
executable = /usr/libexec/dovecot/imap-login
group = dovenull
process_min_avail = 2
service_count = 0
vsz_limit = 1 G
}
service imap {
executable = /usr/libexec/dovecot/imap
process_limit = 8000
process_min_avail = 12
vsz_limit = 1 G
}
service managesieve-login {
executable = /usr/libexec/dovecot/managesieve-login
inet_listener sieve {
port = 2000
}
process_limit = 2000
vsz_limit = 1 G
}
service managesieve {
executable = /usr/libexec/dovecot/managesieve
process_limit = 5000
}
service pop3-login {
executable = /usr/libexec/dovecot/pop3-login
process_limit = 5000
process_min_avail = 2
service_count = 0
vsz_limit = 1 G
}
service pop3 {
executable = /usr/libexec/dovecot/pop3
process_limit = 5000
}
service stats {
fifo_listener stats-mail {
mode = 0600
user = entrega
}
}
ssl_ca =
ssl_cert =
ssl_cipher_list = ECDHE-RSA-AES128-SHA256:AES128-GCM-SHA256:RC4:HIGH:!MD5:!aNULL:!EDH
ssl_dh = # hidden, use -P to show it
ssl_key = # hidden, use -P to show it
userdb {
args = /etc/dovecot/dovecot-ldap.conf
driver = ldap
}
userdb {
args = /etc/dovecot/dovecot-ldap-userdb.conf
driver =
