Panic mbox_sync_handle_header
Hello all, I'm getting a very often panic on my local dovecot setup when accessing some of the local folders. Error message is always Oct 25 14:34:50 imap(irusskikh): Panic: file mbox-sync.c: line 665 (mbox_sync_handle_header): assertion failed: (mail_ctx->mail.from_offset + move_diff != 1 && mail_ctx->mail.from_offset + move_diff != 2) Oct 25 14:34:50 imap(irusskikh): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xa05a2) [0x7f05e40005a2] -> /usr/lib/dovecot/libdovecot.so.0(+0xa069a) [0x7f05e400069a] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f05e3f901d8] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x8b1d7) [0x7f05e432b1d7] -> /usr/lib/dovecot/libdovecot-storage.so.0(mbox_sync+0x48) [0x7f05e432bbf8] -> /usr/lib/dovecot/libdovecot-storage.so.0(mbox_storage_sync_init+0x3b) [0x7f05e432bd3b] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x44) [0x7f05e42eb704] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x37) [0x7f05e42eb7a7] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_storage_get_status+0x31) [0x7f05e43663c1] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_get_status+0x31) [0x7f05e42ebb51] -> dovecot/imap(imap_status_get+0x77) [0x7f05e4c25017] -> dovecot/imap(cmd_status+0x19e) [0x7f05e4c171de] -> dovecot/imap(command_exec+0x5c) [0x7f05e4c1ce0c] -> dovecot/imap(+0x1b382) [0x7f05e4c1b382] -> dovecot/imap(+0x1b41c) [0x7f05e4c1b41c] -> dovecot/imap(client_handle_input+0x18d) [0x7f05e4c1b7dd] -> dovecot/imap(client_input+0xac) [0x7f05e4c1bd2c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x52) [0x7f05e4016642] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x12e) [0x7f05e4017cee] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x36) [0x7f05e40166d6] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f05e4016888] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f05e3f9aa83] -> dovecot/imap(main+0x329) [0x7f05e4c0e139] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) [0x7f05e3b81b97] -> dovecot/imap(_start+0x2a) [0x7f05e4c0e2ca] Oct 25 14:34:50 imap(irusskikh): Fatal: master: service(imap): child 12785 killed with signal 6 (core dumped) # 2.2.33.2 (d6601f4ec): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.21 (92477967) # OS: Linux 4.4.0-17763-Microsoft x86_64 Ubuntu 18.04.4 LTS Running now on WSL, but have the same badness on generic ubuntu setup as well. Could somebody guide me if thats something known, or me doing bad things? I'm not on the list, so please CC me. Igor
Re: doveadm SSL problem with recent update
Citeren "Trever L. Adams" : With a recent update, I started seeing this: doveconf: Fatal: Error in configuration file /etc/dovecot/conf.d/10-ssl.conf line 14: ssl_cert: Can't open file /etc/letsencrypt/live/SERVER/fullchain.pem: Permission denied 1 4 * * * vmail /usr/bin/doveadm expunge -A mailbox MAILBOXINQUESTION savedbefore 1w is one of the crontab entries I am seeing this for. Is there an option to keep doveadm from trying to use the ssl cert from that configuration file? I do not want to have the vmail user having access to the pem cert. Rename your existing 10-ssl.conf to 10-ssl.conf.ext and make it readable by root only. Now create a new 10-ssl.conf file with the following content: # SSL/TLS support: yes, no, required. ssl = no !include_try 10-ssl.conf.ext This will cause all non-root users to disable SSL and will only enable it when started as root.
forwarding email with sieve of spf domains
Say someone has setup spf for his domain and sends an email to a user that has in roundcube enabled the sieve forward. If the message is forwarded without altering the message headers, this could result in a message being blocked or not relayed, because sending hosts ip, is not in the spf of the from: domain. Possible solutions are: - add option if enabled, it replaces the From: with that of the email address of the sieve user. (Maybe move the original sender to the Reply-To header? Maybe exception for 'internal' forward?) - Upon processing the message, check the spf records, if they are enforced, do the above, otherwise do nothing. https://tools.ietf.org/html/rfc5228#section-4.2
Re: forwarding email with sieve of spf domains
There's no ambiguity here, if you send a message, you are the sender. The envelope from should be yours. On Sunday, 25/10/2020 at 11:48 Marc Roos wrote: Say someone has setup spf for his domain and sends an email to a user that has in roundcube enabled the sieve forward. If the message is forwarded without altering the message headers, this could result in a message being blocked or not relayed, because sending hosts ip, is not in the spf of the from: domain. Possible solutions are: - add option if enabled, it replaces the From: with that of the email address of the sieve user. (Maybe move the original sender to the Reply-To header? Maybe exception for 'internal' forward?) - Upon processing the message, check the spf records, if they are enforced, do the above, otherwise do nothing. https://tools.ietf.org/html/rfc5228#section-4.2
Dovecot replication not picking up new mail in maildir
I've got a basic dovecot master-master replication setup between two
servers (imap1 and imap2). Actions such as deleting messages or moving
them between folders are replicated correctly and quickly.
However, when new mail is delivered by postfix, replication does not
occur until something else triggers it, such as one of the delete/move
actions mentioned earlier. A search suggested this may be due to the
notify plugin, but I don't see any issue with the settings. Output of
doveconf -n:
# 2.2.36 (1f10bfa63): /etc/dovecot/dovecot.conf
# OS: Linux 4.19.118-0-vanilla x86_64 CentOS Linux release 7.8.2003 (Core)
# Hostname: imap1
disable_plaintext_auth = no
doveadm_password = # hidden, use -P to show it
first_valid_uid = 1000
haproxy_trusted_networks = 172.16.0.0/24
mail_location = maildir:~/Mail
mail_plugins = zlib notify replication
mailbox_list_index = yes
mailbox_list_index_include_inbox = yes
maildir_empty_new = yes
mbox_write_locks = fcntl
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
}
passdb {
args = /etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
plugin {
mail_replica = tcp:imap2:9943
replication_sync_timeout = 2
}
protocols = imap
service aggregator {
fifo_listener replication-notify-fifo {
mode = 0666
user = dovecot
}
unix_listener replication-notify {
mode = 0666
user = dovecot
}
}
service config {
unix_listener config {
user = dovecot
}
}
service doveadm {
inet_listener {
port = 9943
}
}
service imap-login {
inet_listener imap_hpx {
haproxy = yes
port = 943
}
}
service replicator {
process_min_avail = 1
unix_listener replicator-doveadm {
mode = 0666
}
}
ssl = no
userdb {
args = /etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
protocol lda {
mail_plugins = zlib notify replication
}
protocol imap {
mail_plugins = zlib notify replication imap_zlib
}
I specifically tried setting the following as no and yes as they can
affect notifications:
mailbox_list_index = yes
mailbox_list_index_include_inbox = yes
maildir_empty_new = yes
The other server has the same settings except obviously mail_replica =
tcp:imap1:9943.
Any suggestions welcome, thanks!
RE: forwarding email with sieve of spf domains
However as far as I can determine from my forward test is that, the from header is currently from the original message and not from the forwarder. -Original Message- From: Scott Q. [mailto:qm...@top-consulting.net] Sent: Sunday, October 25, 2020 5:01 PM To: Marc Roos; dovecot Subject: Re: forwarding email with sieve of spf domains There's no ambiguity here, if you send a message, you are the sender. The envelope from should be yours. On Sunday, 25/10/2020 at 11:48 Marc Roos wrote: Say someone has setup spf for his domain and sends an email to a user that has in roundcube enabled the sieve forward. If the message is forwarded without altering the message headers, this could result in a message being blocked or not relayed, because sending hosts ip, is not in the spf of the from: domain. Possible solutions are: - add option if enabled, it replaces the From: with that of the email address of the sieve user. (Maybe move the original sender to the Reply-To header? Maybe exception for 'internal' forward?) - Upon processing the message, check the spf records, if they are enforced, do the above, otherwise do nothing. https://tools.ietf.org/html/rfc5228#section-4.2
SV: forwarding email with sieve of spf domains
Yes, putting the From: into Reply-To: is a good idea to ensure the reply button in receiver´s client doesn't break. But remember to ONLY do it when Reply-To: is not present. To avoid removing important information from the email, like the original sender, it can be good to always add a header like X-Original-Sender with the original from: when rewriting in this way. OTOH I think this type of rewriting should be done in the MTA that is responsible for sending the email off the server, NOT in dovecot/sieve. In exim theres already built-in support for this type of rewriting, and I have such rewriting on all domains for which are forward-only - to avoid SPF errors. -Ursprungligt meddelande- Från: dovecot-boun...@dovecot.org För Marc Roos Skickat: den 25 oktober 2020 16:49 Till: dovecot Ämne: forwarding email with sieve of spf domains Say someone has setup spf for his domain and sends an email to a user that has in roundcube enabled the sieve forward. If the message is forwarded without altering the message headers, this could result in a message being blocked or not relayed, because sending hosts ip, is not in the spf of the from: domain. Possible solutions are: - add option if enabled, it replaces the From: with that of the email address of the sieve user. (Maybe move the original sender to the Reply-To header? Maybe exception for 'internal' forward?) - Upon processing the message, check the spf records, if they are enforced, do the above, otherwise do nothing. https://tools.ietf.org/html/rfc5228#section-4.2 smime.p7s Description: S/MIME Cryptographic Signature
SV: forwarding email with sieve of spf domains
He of course meant the From: MIME sender. This can fail in a SPF check if identity aligment is set to strict. Rewriting the From: heasder is one way to solve it, another way which preserves the original message in full, is to encapsulate the original message ina new message/rfc822 container where the new outer container, does have the rewritten details. Från: dovecot-boun...@dovecot.org För Scott Q. Skickat: den 25 oktober 2020 17:02 Till: Marc Roos ; dovecot Ämne: Re: forwarding email with sieve of spf domains There's no ambiguity here, if you send a message, you are the sender. The envelope from should be yours. On Sunday, 25/10/2020 at 11:48 Marc Roos wrote: Say someone has setup spf for his domain and sends an email to a user that has in roundcube enabled the sieve forward. If the message is forwarded without altering the message headers, this could result in a message being blocked or not relayed, because sending hosts ip, is not in the spf of the from: domain. Possible solutions are: - add option if enabled, it replaces the From: with that of the email address of the sieve user. (Maybe move the original sender to the Reply-To header? Maybe exception for 'internal' forward?) - Upon processing the message, check the spf records, if they are enforced, do the above, otherwise do nothing. https://tools.ietf.org/html/rfc5228#section-4.2 smime.p7s Description: S/MIME Cryptographic Signature
Re: forwarding email with sieve of spf domains
Am 25.10.20 um 16:48 schrieb Marc Roos: Say someone has setup spf for his domain and sends an email to a user that has in roundcube enabled the sieve forward. If the message is forwarded without altering the message headers, this could result in a message being blocked or not relayed, because sending hosts ip, is not in the spf of the from: domain. Possible solutions are: - add option if enabled, it replaces the From: with that of the email address of the sieve user. (Maybe move the original sender to the Reply-To header? Maybe exception for 'internal' forward?) - Upon processing the message, check the spf records, if they are enforced, do the above, otherwise do nothing. https://tools.ietf.org/html/rfc5228#section-4.2 long time ago i did tests about this https://blog.sys4.de/email-forward-mit-sieve-ohne-spf-dmarc-und-dkim-konflikte-de.html perhaps it helps , but its not up2date -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Schleißheimer Straße 26/MG, 80333 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
Looking for a guide to collect all e-mail from the ISP mail server
Hi all:
I am evaluating mail server solutions for a small business. The trouble is, I
am only a part-time admin and a newbie to mail servers.
Most guides I have seen are rather unrealistic: they encourage you to expose your e-mail server to the Internet, and hope that you have the resources
to keep it patched up.
I would rather have an internal mail server that collects e-mails from a standard ISP mail server. It is like the old "POP3 Connector" that came with
Microsoft Exchange. Sometimes, there is a mailbox per user on the ISP, and a corresponding one on the local server. Other times, there is a single
"catch all" or "multidrop" mailbox on the ISP.
Users can still access their internal mailboxes from outside through an OpenVPN connection. The goal is that only VPN, and perhaps SSH, are
accessible from the outside. We do not need to arrange any special SMTP configuration with the ISP either.
This kind of mail server setup is rather different to the standard configuration. You do not normally need you own antivirus and spam filter, and you
do not need to configure SSL certificates, MX or SPF DNS records. Most ISP handle that correctly and economically. Internal e-mail does not leave
your LAN, and your internal SMTP server is just a relay for the external ISP SMTP server.
Furthermore, most guides do not explain how to setup an autoresponder ("I am on holiday until xxx") so that users can enable theirs with the mouse.
Editing configuration files over SSH is not really an option for normal users. This detail is important because it could be the only thing I need
above standard e-mail. Further groupware features can be seen as nice but ultimately unnecessary luxury, and a basic shared calendar can be
accomplished with a separate server like https://radicale.org/ and a calendar client like one built into Thunderbird. Hopefully, that is all I would
need for a small business.
Can anyone point me to the kind of guide I need? Failing that, I would need information or examples about using fetchmail, getmail or similar software
with Dovecot. Good or bad experiences from you guys would also help.
Each of those tools has a detailed man page, but there are many options and ways with different advantages and disadvantages. I would need a simpler
guide to get started.
I am aware that there are pre-packaged mail server solutions that would perhaps bring an easy-to-use autoresponder, but I haven't seen one yet that
where you could tick a box like "this server is only internal and collects mail from the ISP server" during installation. Nor have I seen instructions
about reconfiguring the mail server for my ISP mail scenario.
I am prepared to learn more and write my own Perl scripts and/or installation guide, but it would be stupid to waste time if something easy already
exists. After all, the setup I am describing (external ISP mail server + internal mail server) is not so weird.
Thanks in advance,
rdiez
Re: forwarding email with sieve of spf domains
Marc Roos skrev den 2020-10-25 16:48: Say someone has setup spf for his domain and sends an email to a user that has in roundcube enabled the sieve forward. roundcube does change envelope from on forwarding emails, if thats not the case i like to see logs proving it does not sorry if it miss that in sieve rules https://tools.ietf.org/html/rfc5228#section-4.2 oh well lets talk about dmarc now, where srs is not supported, likewise sid-milter still do sender-id tracking we are in 2020 now
RE: Looking for a guide to collect all e-mail from the ISP mail server
Maybe get something like Zimbra, such solutions also have support that
you can buy when you need it or don't have time (I guess).
-Original Message-
From: R. Diez [mailto:rdiezmail-2...@yahoo.de]
Sent: Sunday, October 25, 2020 6:57 PM
To: dovecot@dovecot.org
Subject: Looking for a guide to collect all e-mail from the ISP mail
server
Hi all:
I am evaluating mail server solutions for a small business. The trouble
is, I am only a part-time admin and a newbie to mail servers.
Most guides I have seen are rather unrealistic: they encourage you to
expose your e-mail server to the Internet, and hope that you have the
resources
to keep it patched up.
I would rather have an internal mail server that collects e-mails from a
standard ISP mail server. It is like the old "POP3 Connector" that came
with
Microsoft Exchange. Sometimes, there is a mailbox per user on the ISP,
and a corresponding one on the local server. Other times, there is a
single
"catch all" or "multidrop" mailbox on the ISP.
Users can still access their internal mailboxes from outside through an
OpenVPN connection. The goal is that only VPN, and perhaps SSH, are
accessible from the outside. We do not need to arrange any special SMTP
configuration with the ISP either.
This kind of mail server setup is rather different to the standard
configuration. You do not normally need you own antivirus and spam
filter, and you
do not need to configure SSL certificates, MX or SPF DNS records. Most
ISP handle that correctly and economically. Internal e-mail does not
leave
your LAN, and your internal SMTP server is just a relay for the external
ISP SMTP server.
Furthermore, most guides do not explain how to setup an autoresponder
("I am on holiday until xxx") so that users can enable theirs with the
mouse.
Editing configuration files over SSH is not really an option for normal
users. This detail is important because it could be the only thing I
need
above standard e-mail. Further groupware features can be seen as nice
but ultimately unnecessary luxury, and a basic shared calendar can be
accomplished with a separate server like https://radicale.org/ and a
calendar client like one built into Thunderbird. Hopefully, that is all
I would
need for a small business.
Can anyone point me to the kind of guide I need? Failing that, I would
need information or examples about using fetchmail, getmail or similar
software
with Dovecot. Good or bad experiences from you guys would also help.
Each of those tools has a detailed man page, but there are many options
and ways with different advantages and disadvantages. I would need a
simpler
guide to get started.
I am aware that there are pre-packaged mail server solutions that would
perhaps bring an easy-to-use autoresponder, but I haven't seen one yet
that
where you could tick a box like "this server is only internal and
collects mail from the ISP server" during installation. Nor have I seen
instructions
about reconfiguring the mail server for my ISP mail scenario.
I am prepared to learn more and write my own Perl scripts and/or
installation guide, but it would be stupid to waste time if something
easy already
exists. After all, the setup I am describing (external ISP mail server
+ internal mail server) is not so weird.
Thanks in advance,
rdiez
Re: Looking for a guide to collect all e-mail from the ISP mail server
Hello R,
Your goal does not sound weird. The most painless way might be to fetch
incoming messages from the ISP's IMAP and deliver them to your local
dovecot. A shortened fetchmailrc would read:
poll remote.server …
user …, password …
folder 'INBOX'
fetchall
idle
ssl
mda "HOME=%T /usr/bin/sudo -u %T /usr/lib/dovecot/deliver"
That way your users can create their vacancies with the ISP portal, the
ISP will do availability, antivirus etc. You can even use sieve on
delivery. Perhaps fetch "Spam" too, if your ISP files it away.
Beware, you have to somehow keep tabs on remote and local usernames.
Passwords will be different. Local updates should be no problem with a
reasonable distro, e.g. the dovecot public repo.
Happy becoming a mail server admin!
Peter
Am 25.10.20 um 18:56 schrieb R. Diez:
Hi all:
I am evaluating mail server solutions for a small business. The trouble
is, I am only a part-time admin and a newbie to mail servers.
Most guides I have seen are rather unrealistic: they encourage you to
expose your e-mail server to the Internet, and hope that you have the
resources to keep it patched up.
I would rather have an internal mail server that collects e-mails from a
standard ISP mail server. It is like the old "POP3 Connector" that came
with Microsoft Exchange. Sometimes, there is a mailbox per user on the
ISP, and a corresponding one on the local server. Other times, there is
a single "catch all" or "multidrop" mailbox on the ISP.
Users can still access their internal mailboxes from outside through an
OpenVPN connection. The goal is that only VPN, and perhaps SSH, are
accessible from the outside. We do not need to arrange any special SMTP
configuration with the ISP either.
This kind of mail server setup is rather different to the standard
configuration. You do not normally need you own antivirus and spam
filter, and you do not need to configure SSL certificates, MX or SPF DNS
records. Most ISP handle that correctly and economically. Internal
e-mail does not leave your LAN, and your internal SMTP server is just a
relay for the external ISP SMTP server.
Furthermore, most guides do not explain how to setup an autoresponder
("I am on holiday until xxx") so that users can enable theirs with the
mouse. Editing configuration files over SSH is not really an option for
normal users. This detail is important because it could be the only
thing I need above standard e-mail. Further groupware features can be
seen as nice but ultimately unnecessary luxury, and a basic shared
calendar can be accomplished with a separate server like
https://radicale.org/ and a calendar client like one built into
Thunderbird. Hopefully, that is all I would need for a small business.
Can anyone point me to the kind of guide I need? Failing that, I would
need information or examples about using fetchmail, getmail or similar
software with Dovecot. Good or bad experiences from you guys would also
help.
Each of those tools has a detailed man page, but there are many options
and ways with different advantages and disadvantages. I would need a
simpler guide to get started.
I am aware that there are pre-packaged mail server solutions that would
perhaps bring an easy-to-use autoresponder, but I haven't seen one yet
that where you could tick a box like "this server is only internal and
collects mail from the ISP server" during installation. Nor have I seen
instructions about reconfiguring the mail server for my ISP mail scenario.
I am prepared to learn more and write my own Perl scripts and/or
installation guide, but it would be stupid to waste time if something
easy already exists. After all, the setup I am describing (external ISP
mail server + internal mail server) is not so weird.
Thanks in advance,
rdiez
Re: Looking for a guide to collect all e-mail from the ISP mail server
You need SPF and DKIM for your outgoing email to be accepted.
My idea of a secure email server is to use submission port 587. Expose port 25
to the world and aggressively filter all remaining email ports with a firewall.
And I mean aggressive. Geographically filter so only countries where youe users
reside can send and retrieve email. Block major hosting IP space.
How many users will be on the system? If you can handle it, assign all the
email passwords. This means you need to contact them out of band. I avoid
cpanel or similar internet access to email settings. I use nothing but ssh to
maintain my server.
Original Message
From: rdiezmail-2...@yahoo.de
Sent: October 25, 2020 10:57 AM
To: dovecot@dovecot.org
Subject: Looking for a guide to collect all e-mail from the ISP mail server
Hi all:
I am evaluating mail server solutions for a small business. The trouble is, I
am only a part-time admin and a newbie to mail servers.
Most guides I have seen are rather unrealistic: they encourage you to expose
your e-mail server to the Internet, and hope that you have the resources
to keep it patched up.
I would rather have an internal mail server that collects e-mails from a
standard ISP mail server. It is like the old "POP3 Connector" that came with
Microsoft Exchange. Sometimes, there is a mailbox per user on the ISP, and a
corresponding one on the local server. Other times, there is a single
"catch all" or "multidrop" mailbox on the ISP.
Users can still access their internal mailboxes from outside through an OpenVPN
connection. The goal is that only VPN, and perhaps SSH, are
accessible from the outside. We do not need to arrange any special SMTP
configuration with the ISP either.
This kind of mail server setup is rather different to the standard
configuration. You do not normally need you own antivirus and spam filter, and
you
do not need to configure SSL certificates, MX or SPF DNS records. Most ISP
handle that correctly and economically. Internal e-mail does not leave
your LAN, and your internal SMTP server is just a relay for the external ISP
SMTP server.
Furthermore, most guides do not explain how to setup an autoresponder ("I am on
holiday until xxx") so that users can enable theirs with the mouse.
Editing configuration files over SSH is not really an option for normal users.
This detail is important because it could be the only thing I need
above standard e-mail. Further groupware features can be seen as nice but
ultimately unnecessary luxury, and a basic shared calendar can be
accomplished with a separate server like https://radicale.org/ and a calendar
client like one built into Thunderbird. Hopefully, that is all I would
need for a small business.
Can anyone point me to the kind of guide I need? Failing that, I would need
information or examples about using fetchmail, getmail or similar software
with Dovecot. Good or bad experiences from you guys would also help.
Each of those tools has a detailed man page, but there are many options and
ways with different advantages and disadvantages. I would need a simpler
guide to get started.
I am aware that there are pre-packaged mail server solutions that would perhaps
bring an easy-to-use autoresponder, but I haven't seen one yet that
where you could tick a box like "this server is only internal and collects mail
from the ISP server" during installation. Nor have I seen instructions
about reconfiguring the mail server for my ISP mail scenario.
I am prepared to learn more and write my own Perl scripts and/or installation
guide, but it would be stupid to waste time if something easy already
exists. After all, the setup I am describing (external ISP mail server +
internal mail server) is not so weird.
Thanks in advance,
rdiez
Re: Looking for a guide to collect all e-mail from the ISP mail server
Your goal does not sound weird. OK, thanks for the confirmation. The most painless way might be to fetch incoming messages from the ISP's IMAP and deliver them to your local dovecot. A shortened fetchmailrc would read: poll remote.server … user …, password … folder 'INBOX' fetchall idle ssl mda "HOME=%T /usr/bin/sudo -u %T /usr/lib/dovecot/deliver" Brilliant, thanks for the info. That way your users can create their vacancies with the ISP portal, > [...] That's a good idea. But then internal e-mails need to go out to the ISP, don't they? Because, if internal e-mails get delivered locally, the vacation autoresponses on the ISP will not trigger, will they? The trouble is, with that configuration, if the Internet link goes down, internal e-mail stops working too. I was hoping that there would be a complete mail server setup guide somewhere for this kind of setup. But I guess I'll have to piece all these information snippets together. Regards, rdiez
Re: Looking for a guide to collect all e-mail from the ISP mail server
You need SPF and DKIM for your outgoing email to be accepted. > [...] I don't understand why that is the case (but keep in mind that I am a newbie). Is it not possible to set up some internal SMTP server that only relies the e-mails to the external ISP SMTP server? The internal SMTP server would then act like a normal user's Thunderbird. At first I tought that the internal SMTP server would need to know the password for each mailbox user. But then I asked, and the ISP SMTP server allegedly accepts any source e-mail address, as long as you are using one e-mail account that is valid in the domain. I wonder if that is standard practice. My idea of a secure email server is to use submission port 587. Expose port 25 to the world and aggressively filter all remaining email ports with a firewall. And I mean aggressive. Geographically filter so only countries where youe users reside can send and retrieve email. Block major hosting IP space. Geo blocking can be problematic. Depending on the small business, some customers and suppliers may sit in China or some other geographical area you would normally block. I am too afraid, I would not expose any such port on the Internet. Who knows if the mail server stays months without an update. If I am to recommend or implement any such mail server solution to a small business, I would insist that the e-mail server is not exposed at all on the Internet. A web interface etc. is not a problem: I just connect with a VPN and bypass most external security issues. If you are the admin, you can also forward the web interface over an SSH connection. Best regards, rdiez
Re: Looking for a guide to collect all e-mail from the ISP mail server
Hello R, reply inline below: Am 25.10.20 um 23:12 schrieb R. Diez: That way your users can create their vacancies with the ISP portal, > [...] That's a good idea. But then internal e-mails need to go out to the ISP, don't they? Because, if internal e-mails get delivered locally, the vacation autoresponses on the ISP will not trigger, will they? The trouble is, with that configuration, if the Internet link goes down, internal e-mail stops working too. Hello R, I only wrote about the incoming side - of course, you also want to send mail to remote users, and that includes users with an address of …@myisp.com. They will go to the ISP and be fetched to local from there. And if internet's down, e-mail will stop working anyways, so why bother? Even facebook/whatsupp will stop working then! With some tinkering, you can configure your local relay smtp to deliver those locally, but if your people do not talk about their vacancies over the water cooler, then they will miss that reminder then. I was hoping that there would be a complete mail server setup guide somewhere for this kind of setup. But I guess I'll have to piece all these information snippets together. Sorry, the world is too big :) -- peter
Re: Looking for a guide to collect all e-mail from the ISP mail server
You look spammy if you don't have SPF or DKIM, and hopefully both. Your email will either be bounced or sent to a spam folder. You need a reverse pointer as well, but that shouldn't be an issue. The situation is actually worse than it sounds. ATT/SBC needs to whitelist you by IP if you are using a VPS. Spectrum/Charter just plain blocks many VPS with no recourse. Regarding geofencing, look back at my post. I leave port 25 open to the world. I can receive email from any country. Using submission port 587 means you can geofence from where your employee sends and receives email. It does not effect your customers since they use port 25. The reason I run my own email server is I got hacked when using a hosting service. The hacker used a vulnerability in RoundCube and could send email as me. My PayPal account password was then changed. The hacker was in Morocco. I'm sure Morocco is a fine country but I don't plan on visiting it and thus don't need to access my email from there. Note the hacker could have changed my email password too but didn't. To top it off, I don't even use RoundCube. Never use a browser for email. When I set up my own email / webserver I made it a point to not use any GUI control panel. If there is no hook to change a password from a control panel then it won't happen. You reduce the attack surface. All passwords are SHA512. You geofence all email ports except 25. I also have a VPS using openvpn but it is on a different IP. That is a tunnel out of it to use the internet. Now I think for what you want to do is to have openvpn show up as the local host. What you might want to do is join the postfix users group. I wouldn't bring up this kind of proxied email scheme you want to set up. Rather just ask if it is possible to set up postfix/dovecot so that the user who will always be on a VPN can send and receive email. That is I think it will boil down to permit local host and nothing else in certain places. There are guru status users there. One thing you will learn about email servers is there are many programs to chain together. However think of light bulbs in series. The more in the chain, the more likely it is to fail. I dropped SpamAssassin and amavisd due to poor reliability. That was when I used freeBSD. I now run centos but just don't bother with those extra programs. I use RBLs for spam blocking. I use my brain for antivirus. Antivirus isn't all that good anyway. The key with antivirus is at what point in time do they recognize the file is a virus. I send all my malware links to virus total.com and maybe two will recognize the link goes to malware. Original Message From: rdiezmail-2...@yahoo.de Sent: October 25, 2020 3:25 PM To: li...@lazygranch.com Cc: dovecot@dovecot.org Subject: Re: Looking for a guide to collect all e-mail from the ISP mail server > You need SPF and DKIM for your outgoing email to be accepted. > [...] I don't understand why that is the case (but keep in mind that I am a newbie). Is it not possible to set up some internal SMTP server that only relies the e-mails to the external ISP SMTP server? The internal SMTP server would then act like a normal user's Thunderbird. At first I tought that the internal SMTP server would need to know the password for each mailbox user. But then I asked, and the ISP SMTP server allegedly accepts any source e-mail address, as long as you are using one e-mail account that is valid in the domain. I wonder if that is standard practice. > My idea of a secure email server is to use submission port 587. > Expose port 25 to the world and aggressively filter all remaining > email ports with a firewall. And I mean aggressive. Geographically filter > so only countries where youe users reside can send and retrieve email. > Block major hosting IP space. Geo blocking can be problematic. Depending on the small business, some customers and suppliers may sit in China or some other geographical area you would normally block. I am too afraid, I would not expose any such port on the Internet. Who knows if the mail server stays months without an update. If I am to recommend or implement any such mail server solution to a small business, I would insist that the e-mail server is not exposed at all on the Internet. A web interface etc. is not a problem: I just connect with a VPN and bypass most external security issues. If you are the admin, you can also forward the web interface over an SSH connection. Best regards, rdiez
Re: Looking for a guide to collect all e-mail from the ISP mail server
At 25 October, 2020 R. Diez wrote: > > I am too afraid, I would not expose any such port on the Internet. Who knows > if the mail server stays months without an update. If I am to recommend or > implement any such mail server solution to a small business, I would insist > that the e-mail server is not exposed at all on the Internet. Setting and forgetting any server/service to run unpatched for months is generally a bad idea. I presume that you won't be maintaining this for them long term -- why not just point them at a hosting service like google apps, and let google keep things up to date?
SV: Looking for a guide to collect all e-mail from the ISP mail server
>>"Never use a browser for email." I don't agree. In fact, using a browser for email or atleast initial setup, is actually more secure. This because SMTP/IMAP clients normally don't support 2FA, so you would have to "hack" a solution to enable 2FA for email. This can be made in 2 ways: Either, you have a full fledged email setup. Whats important, is, to prevent auth-bypass holes, you remove the authentication in RoundCube or whatever webmail you use, and instead use a reverse-proxy or firewall authentication instead. Thus an unauthenticated user doesn't even touch RoundCube/webmail at all, but must authenticate at a prior stage. The second way, is to not have webmail at all, but instead have a authentication gateway in browser, where you must auth with 2FA and captcha. The only purpose of this gateway, is to authenticate users with 2FA before their IP is whitelisted. After this, you simply have a script, that upon valid login (with 2FA) in either webmail or auth gateway, you set the authorized IP of the user to this. Whats happen then, is that each account will have an authorized IP attached (you could limit it to the /24 to cater for mobile clients), and then login to that account, will only be accepted from that authorized IP. This then allows SMTP/IMAP usage from that IP. If you want to go even more secure, you could restrict the firewall to the list of all IPs that all users have dynamically, and then in the SMTP/IMAP server, lock down auth to the authorized IP of that particular user account only. Its very important, that upon authing with a incorrect IP, that the server responds in the same way as a invalid password was specified, in this way, if someone attempts to bruteforce the password, they will "miss" the correct password, if the server does not react differently to a correct password but invalid IP. Thus bots that bruteforce will not gain any success. All this can be combined with permanent whitelists and geoIP whitelists, to avoid users having to authenticate with 2FA for "trusted" locations. One example would be to have the local office as permanent whitelist, and also have it that any IP in the user's "home country" is permanently whitelisted for his account once the user authenticates with 2FA. Other IPs outside his home country, is then only whitelisted once, next 2FA login, the old whitelist is simply deleted. smime.p7s Description: S/MIME Cryptographic Signature
SV: Looking for a guide to collect all e-mail from the ISP mail server
>> why not just point them at a hosting service like google apps, and let google keep things up to date? Costs money, and also the problem is that gmail imposes heavy spam filters and "reputation blocks" meaning smaller providers with low email volumes, are put in the spam folder, even if they never send spam, just because their email volume is so low (ergo, they must prove they don't spam before getting out of ispam folder) Another thing is that you cannot impose IP restrictions when using Google Apps, or have SSO with trusted access from inside the office. (for example - scan your badge at the office door, your personal computer is automatically logged on and you get access to everything). With locally hosted servers, of course you have to keep them updated. Most linux distributions can keep them updated automatically. smime.p7s Description: S/MIME Cryptographic Signature
Re: SV: Looking for a guide to collect all e-mail from the ISP mail server
Good luck with all that coding. I have four years now of running my own email server. Zero hacks. I keep the attack surface to a minimum. Less is more. One thing you don't want to do is write your own code. This stuff is always way harder than you think. Worse yet you run alpha generation code because you are the only one using it. All software has bugs. What you need is a mass of users flogging the code and finding the bugs. Now if you do use a browser, you have to deal with leaks, bugs, possible process interaction if more than one tab is open, and possibly browser extensions hacks if extensions are used. Count me out. And did you miss the part where I was hacked via RoundCube? Original Message From: sebast...@sebbe.eu Sent: October 25, 2020 9:47 PM To: dovecot@dovecot.org Reply-to: dovecot@dovecot.org Subject: SV: Looking for a guide to collect all e-mail from the ISP mail server >>"Never use a browser for email." I don't agree. In fact, using a browser for email or atleast initial setup, is actually more secure. This because SMTP/IMAP clients normally don't support 2FA, so you would have to "hack" a solution to enable 2FA for email. This can be made in 2 ways: Either, you have a full fledged email setup. Whats important, is, to prevent auth-bypass holes, you remove the authentication in RoundCube or whatever webmail you use, and instead use a reverse-proxy or firewall authentication instead. Thus an unauthenticated user doesn't even touch RoundCube/webmail at all, but must authenticate at a prior stage. The second way, is to not have webmail at all, but instead have a authentication gateway in browser, where you must auth with 2FA and captcha. The only purpose of this gateway, is to authenticate users with 2FA before their IP is whitelisted. After this, you simply have a script, that upon valid login (with 2FA) in either webmail or auth gateway, you set the authorized IP of the user to this. Whats happen then, is that each account will have an authorized IP attached (you could limit it to the /24 to cater for mobile clients), and then login to that account, will only be accepted from that authorized IP. This then allows SMTP/IMAP usage from that IP. If you want to go even more secure, you could restrict the firewall to the list of all IPs that all users have dynamically, and then in the SMTP/IMAP server, lock down auth to the authorized IP of that particular user account only. Its very important, that upon authing with a incorrect IP, that the server responds in the same way as a invalid password was specified, in this way, if someone attempts to bruteforce the password, they will "miss" the correct password, if the server does not react differently to a correct password but invalid IP. Thus bots that bruteforce will not gain any success. All this can be combined with permanent whitelists and geoIP whitelists, to avoid users having to authenticate with 2FA for "trusted" locations. One example would be to have the local office as permanent whitelist, and also have it that any IP in the user's "home country" is permanently whitelisted for his account once the user authenticates with 2FA. Other IPs outside his home country, is then only whitelisted once, next 2FA login, the old whitelist is simply deleted.
Re: Directors and LMTP: mailAlternateAddress of same user
Dear Ioannis,
> I've searched around for a solution to this, and figured out that a special
> userdb {} section in the protocol lmtp {} section, with driver = ldap etc etc
> so that i 'd be able to "show" the correct user to the lmtp might work, but
> strangely that userdb {} setting looks like being ignored by lmtp.
A very similar issue was discussed on IRC last Friday. userdb is indeed
ignored on the director, as the different process are going through the
special "director-userdb", which has nothing to do with the userdb you have
define... The documentation
(https://doc.dovecot.org/configuration_manual/authentication/user_databases_userdb/)
was updated to make it cleaner.
The solution is to use passdb instead. Use basically the same configuration as
the one you have for userdb but for passdb. For the ldap configuration itself
replace user_* by pass_*.
I hope that helps,
Vincent
signature.asc
Description: OpenPGP digital signature
