Re: Strategies for protecting IMAP (e.g. MFA)
On Sun, 14 Nov 2021, Michael Peddemors wrote: And there are RBL's now for know IP(s) used by IMAP hackers, including SpamRats RATS-AUTH that can assist in reducing those attacks. These guys also lists brute forcers: http://www.blocklist.de/en/rbldns.html I don't know how well they catch IMAP hackers, but they list 95%+ of our ssh brute forcing attacks. Joseph Tam
Subfolder in sieve not working as expected
On 15-11-2021 23:04, dove...@ptld.com wrote: On 11-15-2021 3:46 pm, Kees van Vloten wrote: I am trying to move incoming mails into subfolders with this sieve script: require ["fileinto", "variables", "mailbox"]; if header :matches "Delivered-To" "*@*" { fileinto :create "INBOX/${2}"; } I am not using variables and my folders are pre-created. But just to give you a starting point, this is what works using maildir++ format with default directory layout. https://doc.dovecot.org/configuration_manual/mail_location/Maildir/ require ["fileinto"]; # rule:[Dovecot] if header :contains "sender" "@dovecot.org" { fileinto "INBOX.Dovecot"; } I have seen that working in the past :-) I switched to LAYOUT=fs recently because of the limitations of the dot as a folder separator (now I can have the domain-name as a folder-name). I found a nice description about folder and namespace issues here: https://forum.hestiacp.com/t/character-not-allowed-in-mailbox-name/566/4 But somehow the sieve implementation does not seem to work properly with LAYOUT=fs. What a good approach to further analyze this issue?
Re: Subfolder in sieve not working as expected
> On 11-15-2021 3:46 pm, Kees van Vloten wrote: > > I am trying to move incoming mails into subfolders with this sieve script: > > require ["fileinto", "variables", "mailbox"]; > if header :matches "Delivered-To" "*@*" > { > fileinto :create "INBOX/${2}"; > } I am not using variables and my folders are pre-created. But just to give you a starting point, this is what works using maildir++ format with default directory layout. https://doc.dovecot.org/configuration_manual/mail_location/Maildir/ require ["fileinto"]; # rule:[Dovecot] if header :contains "sender" "@dovecot.org" { fileinto "INBOX.Dovecot"; }
Subfolder in sieve not working as expected
Hi everybody, I am trying to move incoming mails into subfolders with this sieve script: require ["fileinto", "variables", "mailbox"]; if header :matches "Delivered-To" "*@*" { fileinto :create "INBOX/${2}"; } On a message to u...@example.com I would expect it to be stored in the folder 'example.com' under 'INBOX', but instead a folder named 'INBOX\2fexample.com' is created. I tried several options: - replace / with \/ or \\/ - replace / with . - disable listescape in dovecot.conf Unfortunately nothing did the trick. Any ideas what could fix this? dovecot -n # 2.3.13 (89f716dc2): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.13 (cdd19fe3) # OS: Linux 5.10.0-9-amd64 x86_64 Debian 11.1 # Hostname: servers.example.com auth_default_realm = EXAMPLE.COM auth_gssapi_hostname = servers.example.com auth_krb5_keytab = /etc/keytab/dovecot.keytab auth_master_user_separator = * auth_mechanisms = gssapi gss-spnego plain auth_realms = EXAMPLE.COM first_valid_gid = 986 first_valid_uid = 990 imap_client_workarounds = tb-extra-mailbox-sep login_greeting = Dovecot ready. mail_debug = yes mail_gid = 986 mail_location = maildir:%h/%d/%n/store:LAYOUT=fs:FULLDIRNAME=0_FolderContent:UTF-8:INDEX=%h/%d/%n/index:CONTROL=%h/%d/%n/control:VOLATILEDIR=%h/%d/%n/volatile mail_plugins = zlib notify push_notification listescape acl mail_privileged_group = mail mail_shared_explicit_inbox = yes mail_uid = 990 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext vacation-seconds spamtest imapflags notify imapsieve vnd.dovecot.imapsieve namespace inbox { inbox = yes list = yes mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Inbox { auto = subscribe } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = ~ subscriptions = yes type = private } passdb { args = username_format=%u /etc/dovecot/master-users driver = passwd-file master = yes } passdb { args = /etc/dovecot/ldap_user_to_principal.conf.ext driver = ldap pass = yes } passdb { driver = pam } plugin { acl = vfile:/var/lib/dovecot/global-acls:cache_secs=1 acl_shared_dict = file:/var/lib/dovecot/shared-mailboxes autocreate = Trash autocreate2 = Drafts autocreate3 = Sent autosubscribe = Trash autosubscribe2 = Drafts autosubscribe3 = Sent imapsieve_mailbox1_before = file:/var/lib/dovecot/sieve/imap/learn-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Junk imapsieve_mailbox2_before = file:/var/lib/dovecot/sieve/imap/learn-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Junk imapsieve_mailbox2_name = * listescape_char = \ sieve = file:%h/%d/%n/sieve/sieve;active=%h/%d/%n/sieve/active.sieve sieve_after = /var/lib/dovecot/sieve/after sieve_before = /var/lib/dovecot/sieve/before sieve_default = /var/lib/dovecot/sieve/default.sieve sieve_extensions = +vacation-seconds +reject +notify +imapflags +spamtest sieve_global_extensions = +vnd.dovecot.pipe sieve_pipe_bin_dir = /var/lib/dovecot/sieve/bin sieve_plugins = sieve_imapsieve sieve_extprograms sieve_vacation_default_period = 1d sieve_vacation_max_period = 1d sieve_vacation_min_period = 10s sieve_vacation_use_original_recipient = yes } postmaster_address = sysad...@cvanvloten.nl protocols = " imap lmtp sieve submission" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service imap-login { inet_listener imap { port = 0 } inet_listener imaps { port = 993 ssl = yes } process_min_avail = 1 } service imap { executable = imap } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service submission-login { inet_listener submission { port = 465 ssl = yes } } ssl = required ssl_cert = mail_plugins = zlib notify push_notification listescape acl imap_acl imap_sieve } - Kees
Re: Strategies for protecting IMAP (e.g. MFA)
> Probably because it can be complex to set up and maintain Such is mail and many other things, which falls on the admins. I see no harm in exploring what's what before deciding. > and more would be gained by educating users Yes, users are the weakest link. Users are also the biggest challenge. Making them use 2FA on their phone? Easy. Retraining them to think more like me? Very difficult (assuming you get management buy-in). Again, I'm trying to explore deploying an all "bells and whistles" mail stack that can cater to both an enthusiast environment (like myself) and one in a commercial/enterprise environment. Not having 2FA like I could get from cloud offerings may be a deal breaker for me. > I personally find your style of quoting a bit irritating, especially because you leave the whole original message intac Gmail loves hiding the previous text with an ellipsis, making me not realize. > Why not quote in a manner we have successfully used for a couple of decades? I've just recently started using mailing lists, so replying is still a bit awkward to me. (Probably be easier if we'd use forums.)
imap_metadata plugin panic
Hello, This may be covered somewhere but recently I enabled the metadata plugin to work with sieve as part of some updates I did a while back and I hadn't checked my logs for any issues with it and so far they're not completely taking my system down. Right now, upon trying to delete a folder I noticed the following in my logs... what am I missing? I see the broken SQL query, but I don't know enough about the dict system or the metadata plugin to know how to add the missing information or fix it otherwise. Regards, Elisamuel Resto Nov 15 12:19:19 wyvern dovecot[461]: dict(51438): Panic: lib-sql: Too many bind args (2) for statement: SELECT meta_key FROM metadata WHERE meta_key LIKE AND username = ? Nov 15 12:19:19 wyvern dovecot[461]: dict(51438): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x43) [0x7f449789d073] -> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x20) [0x7f449789d190] -> /usr/lib/dovecot/libdovecot.so.0(+0xfaf1f) [0x7f44978a9f1f] -> /usr/lib/dovecot/libdovecot.so.0(+0xfafb1) [0x7f44978a9fb1] -> /usr/lib/dovecot/libdovecot.so.0(+0x4cd20) [0x7f44977fbd20] -> dovecot/dict [0 clients, 0 lookups:0/0/0/0, 0 iters:0/0/0/0, 0 commits:0/0/0/0](+0x875a) [0x555e60d7775a] -> dovecot/dict [0 clients, 0 lookups:0/0/0/0, 0 iters:0/0/0/0, 0 commits:0/0/0/0](sql_statement_query+0x42) [0x555e60d7f262] -> dovecot/dict [0 clients, 0 lookups:0/0/0/0, 0 iters:0/0/0/0, 0 commits:0/0/0/0](+0xd97f) [0x555e60d7c97f] -> /usr/lib/dovecot/libdovecot.so.0(dict_iterate_values+0x25) [0x7f4497868615] -> dovecot/dict [0 clients, 0 lookups:0/0/0/0, 0 iters:0/0/0/0, 0 commits:0/0/0/0](+0xa929) [0x555e60d79929] -> dovecot/dict [0 clients, 0 lookups:0/0/0/0, 0 iters:0/0/0/0, 0 commits:0/0/0/0](+0xb224) [0x555e60d7a224] -> dovecot/dict [0 clients, 0 lookups:0/0/0/0, 0 iters:0/0/0/0, 0 commits:0/0/0/0](+0xb381) [0x555e60d7a381] -> dovecot/dict [0 clients, 0 lookups:0/0/0/0, 0 iters:0/0/0/0, 0 commits:0/0/0/0](dict_command_input+0xd9) [0x555e60d7a579] -> dovecot/dict [0 clients, 0 lookups:0/0/0/0, 0 iters:0/0/0/0, 0 commits:0/0/0/0](+0x95b8) [0x555e60d785b8] -> /usr/lib/dovecot/libdovecot.so.0(connection_input_default+0x15e) [0x7f44978a16ce] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x6b) [0x7f44978bfebb] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x13b) [0x7f44978c15cb] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x51) [0x7f44978bff61] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x41) [0x7f44978c0131] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x14) [0x7f4497831f74] -> dovecot/dict [0 clients, 0 lookups:0/0/0/0, 0 iters:0/0/0/0, 0 commits:0/0/0/0](main+0x189) [0x555e60d78139] -> /usr/lib/libc.so.6(__libc_start_main+0xd5) [0x7f44972cfb25] -> dovecot/dict [0 clients, 0 lookups:0/0/0/0, 0 iters:0/0/0/0, 0 commits:0/0/0/0](_start+0x2e) [0x555e60d7819e] Nov 15 12:19:19 wyvern dovecot[461]: imap(s...@samresto.dev)<51449><+BGq2NfQM/7Pisr9>: Error: Mailbox Trash/Processed: dict_iterate(priv/c841ad0291c27461ac670100a07d9965/) failed: Connection closed (reply took 0.204 secs (0.204 in dict wait, 0.000 in other ioloops, 0.000 in locks)) Nov 15 12:19:19 wyvern dovecot[461]: dict(51438): Fatal: master: service(dict): child 51438 killed with signal 6 (core dumped) [root@wyvern ~]# dovecot -n # 2.3.16 (7e2e900c1a): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.16 (09c29328) # OS: Linux 5.14.16-arch1-1 x86_64 ext4 # Hostname: wyvern.simplysam.us auth_mechanisms = plain login dict { lastlogin = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext metadata = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext quota_clone = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } doveadm_worker_count = 5 first_valid_gid = 5000 first_valid_uid = 5000 imap_client_workarounds = delay-newmail tb-extra-mailbox-sep tb-lsub-flags last_valid_gid = 5000 last_valid_uid = 5000 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes lda_original_recipient_header = X-Original-To lmtp_client_workarounds = whitespace-before-path lmtp_rcpt_check_quota = yes lmtp_save_to_detail_mailbox = yes mail_attachment_detection_options = add-flags-on-save mail_attribute_dict = proxy::metadata mail_gid = 5000 mail_home = /var/mail/virtual/%Ld/%Ln mail_location = maildir:/var/mail/virtual/%Ld/%Ln mail_plugins = " quota quota_clone mailbox_alias trash fts fts_flatcurve" mail_uid = 5000 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext spamtestplus virustest vnd.dovecot.pipe vnd.dovecot.filter namespace inbox { inbox = yes location = mailbox Archive { auto = subscribe special_use = \Archive } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent {
Re: ZFS storage and backup
On 15.11.21 11:04, James wrote: I will use native ZFS encryption soon. I see no performance issues in test. Don't get hung up on ZFS tuning, mostly ZFS just works. yes I know, I love working with it, I have used it for > 10 years now, but it happened that none of my mailserver projects used ZFS. Regarding storage I tend to use sdbox, from what I have read it seems to be the better option when using a COW filesystem compared to mdbox. One more question is: compression at file system level or in dovecot storage? The reason I am not sure to switch to ssds is that most servers are for non-profit organisations, sports clubs etc. - they also need some storage for pictures, their budget is quite low (so performance testing would only be done out of my interest), and if spinning rust with optimized settings suffices why not. Thanks for your input so far, hope more will come ;-)
[Move mailboxes] 2.2.13 -> 2.3.13: Stuck on certificate verification
Hi, I'm about to move all mailboxes from an old machine - running Dovecot 2.2.13 - to a new machine - running Dovecot 2.3.13 (89f716dc2). Cause the new machine is in a different location I must use SSL encryption. I followed the guide's I found, but I stuck on certificate verification: $ doveadm backup -Ru tcps::12354 doveadm(): Info: Received invalid SSL certificate: unable to get local issuer certificate: /CN= (check ssl_client_ca_* settings?) doveadm(): Error: doveadm server disconnected before handshake: Received invalid SSL certificate: unable to get local issuer certificate: /CN= (check ssl_client_ca_* settings?) doveadm(): Error: Disconnected from remote: Received invalid SSL certificate: unable to get local issuer certificate: /CN= (check ssl_client_ca_* settings?) On port 12354 the server sends an incomplete certificate chain, whereas on port 993 everything is fine. I read that the settings - ssl_client_ca_dir - ssl_client_ca_file are not used on certificate verification for port 12354, one should use the setting ssl_ca Here are the non-default setting on the client side: $ dovecot -n # 2.3.13 (89f716dc2): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.13 (cdd19fe3) # OS: Linux 5.10.0-9-amd64 x86_64 Debian 11.1 ... ssl_ca = .combine.crt ssl_cipher_list = EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH:+AES256:!aNULL:!eNULL:!LOW:!MEDIUM:!3DES:!MD5:!EXP:!PSK:!DSS:!RC4:!SEED:!ECDSA:!CAMELLIA ssl_client_ca_dir = /etc/ssl/certs ssl_client_ca_file = /etc/ssl/certs/ca-certificates.crt ... ssl_min_protocol = TLSv1.2 ssl_prefer_server_ciphers = yes ... verbose_ssl = yes ssl_verify_client_cert = yes According to https://doc.dovecot.org/configuration_manual/dovecot_ssl_configuration/ the setting ssl_ca should contain Issuing CA cert Issuing CA CRL Intermediate CA cert Intermediate CA CRL Root CA cert Root CA CRL But how do I build this file? I tried root certificate, root + intermediate certificate and root + intermediate + signed certificate. None of them made it work... I'm completely stuck on how to make certificate verification work. Can anyone give me a hint? Thanks in advance.
Re: BUG: imapsieve with virtual mailboxes
Hi folks, and @stephanbosch trying to debug the code, even knowing nothing about it, I discovered that @stephanbosch (cc him on this message) fixed the panic error I described in my original message on commit #27ab897f in the pigeonhole project. In this change he only avoided running into the bug if UID of the message could not be determined, which is the case when the target mailbox of a copy or move operation is a virtualbox with a fallback real mailbox. This way the imapsieve is unusable when moving to a virtual mailbox, since what I'm trying to do is a virtualbox for Junk and some others, all with real mailboxes fallbacks and trying to activate the spamassassin learning on this movement, I need to detect when a message is being moved from a junk folder to any other folder and vice-versa. Can anybody help me on making imapsieve to work when dropping messages into virtual mailbox that have a real mailbox fallback? Best regards, Claudemir
Re: Strategies for protecting IMAP (e.g. MFA)
On 15/11/2021 11:52, Arjen de Korte wrote: Citeren Benny Pedersen : On 2021-11-14 20:26, Matthew Richardson wrote: On Sun, 14 Nov 2021 08:12:53 -0800, Michael Peddemors wrote:- And there are RBL's now for know IP(s) used by IMAP hackers, including SpamRats RATS-AUTH that can assist in reducing those attacks. Looking at https://www.spamrats.com/rats-auth.php the "Example Usage in Dovecot" says "PLEASE UPDATE". How would one use a DNSBL like this in Dovecot to reject IMAP connections from listed IPs? submission inet n - y - - smtpd -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_delay_reject=no -o { smtpd_client_restrictions = reject_rbl_client auth.spamrats.com=127.0.0.39, permit } -o { smtpd_relay_restrictions = permit_mynetworks, permit_sasl_authenticated, reject } This is not an answer to the question, this is Postfix syntax. openRelay, dont do it In what way would this create an open relay exactly? The 'permit' at the end of the 'smtpd_client_restrictions' only means that the client is accepted, not that other smtpd restrictions are lifted. resolved version submission inet n - y - - smtpd -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_delay_reject=no -o { smtpd_relay_restrictions = reject_rbl_client auth.spamrats.com=127.0.0.39, permit_mynetworks, permit_sasl_authenticated, reject } Although syntactically correct, it is confusing at best to put client restrictions in another place than smtpd_client_restrictions. Especially with 'smtpd_delay_reject=no' in effect you'd only reject after receiving 'RCPT TO', which is evaluated after 'smtpd_client_restrictions' and 'smtpd_helo_restrictions' during the SMTP transfer. order do matter Indeed. Perhaps I was not clear in my last message. Have a look to this documentation: https://homebox.readthedocs.io/en/latest/email-access-monitoring/ I am available if you have any question to implement something similar yourself. Extending the system to add a second factor authentication is probably easy enough. Kind regards, André -- 𝓐𝓡 - André Rodier
Re: Strategies for protecting IMAP (e.g. MFA)
Citeren Benny Pedersen : On 2021-11-14 20:26, Matthew Richardson wrote: On Sun, 14 Nov 2021 08:12:53 -0800, Michael Peddemors wrote:- And there are RBL's now for know IP(s) used by IMAP hackers, including SpamRats RATS-AUTH that can assist in reducing those attacks. Looking at https://www.spamrats.com/rats-auth.php the "Example Usage in Dovecot" says "PLEASE UPDATE". How would one use a DNSBL like this in Dovecot to reject IMAP connections from listed IPs? submission inet n - y - - smtpd -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_delay_reject=no -o { smtpd_client_restrictions = reject_rbl_client auth.spamrats.com=127.0.0.39, permit } -o { smtpd_relay_restrictions = permit_mynetworks, permit_sasl_authenticated, reject } This is not an answer to the question, this is Postfix syntax. openRelay, dont do it In what way would this create an open relay exactly? The 'permit' at the end of the 'smtpd_client_restrictions' only means that the client is accepted, not that other smtpd restrictions are lifted. resolved version submission inet n - y - - smtpd -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_delay_reject=no -o { smtpd_relay_restrictions = reject_rbl_client auth.spamrats.com=127.0.0.39, permit_mynetworks, permit_sasl_authenticated, reject } Although syntactically correct, it is confusing at best to put client restrictions in another place than smtpd_client_restrictions. Especially with 'smtpd_delay_reject=no' in effect you'd only reject after receiving 'RCPT TO', which is evaluated after 'smtpd_client_restrictions' and 'smtpd_helo_restrictions' during the SMTP transfer. order do matter Indeed.
Re: ZFS storage and backup
On 14/11/2021 14:14, infoomatic wrote: My setups are nothing special with few users, however, I would like to have a nice setup, maybe some of you could contribute to this thread. We are using slow spinning disks, but we may consider using ssds in a not-so-distant future. *) storages: any infos on ZFS options or whether to use mdbox or sdbox, and what configs/options regarding compression etc. OmniOS with ZFS here. I use maildir - just a personal choice and inertia, I have no performance data, no problem and no reason to change. I like being able to see emails as plain files. zfs set compress=gzip and no other changes from default, oh and atime=off on the whole machine. Email gzips well, most other ZFSes I leave on lz4. I say it is better to use the file system to compress rather than getting dovecot to do it. $ zfs get compress,compressratio,used ... NAME PROPERTY VALUE SOURCE .//vmail compressiongzip received .//vmail compressratio 1.82x - .//vmail used 8.55G - 25 mailbox users ("nothing special with few users"). I moved the storage from HDD (mirror plus log) to SSD (mirror) and no one noticed, not even me knowing it had been done and over a local network. I have enough RAM such that repeated reads are cached. I will use native ZFS encryption soon. I see no performance issues in test. Don't get hung up on ZFS tuning, mostly ZFS just works. *) backup: what is a best practice regarding backups? - using only the dovecot tools or leveraging the great features of ZFS (or both) with snapshots etc.? I use automated snapshots and zfs send/receive to a remote backup machine. I auto copy many ZFSes this way so it is minimal effort to do email too. James