Re: smtpd not passing data to rspamd
On Wed, Aug 21, 2019 at 08:06:58PM +, Thomas Smith wrote: > > ? Original Message ? > On Wednesday, August 21, 2019 8:28 AM, Gilles Chehade > wrote: > > > On Wed, Aug 21, 2019 at 03:22:39PM +, Thomas Smith wrote: > > > > > Hi, > > > I've setup filter-rspamd with rspamd. Both appear to be running (smtpd > > > and rspamd), I'm able to query rspamd's controller, access the web UI; > > > smtpd is processing and delivering mail as expected. > > > ps wuax | grep rspam > > > root 86736 0.0 0.4 45236 4008 ?? I 6:30AM 0:00.05 rspamd: main process > > > (rspamd) > > > _rspamd 32135 0.0 1.0 45344 10140 ?? S 6:30AM 0:00.23 rspamd: > > > rspamd_proxy process (localhost:11332) (rspamd) > > > _rspamd 4059 0.0 1.4 45688 14632 ?? S 6:30AM 0:01.63 rspamd: controller > > > process (localhost:11334) (rspamd) > > > _rspamd 16743 0.0 1.1 45384 11020 ?? S 6:30AM 0:00.33 rspamd: normal > > > process (localhost:11333) (rspamd) > > > _smtpd 32851 0.0 0.4 105520 3624 ?? I 6:56AM 0:00.01 > > > /usr/local/bin/filter-rspamd > > > _smtpd 68802 0.0 0.1 844 808 ?? Ip 6:56AM 0:00.00 sh -c > > > /usr/local/bin/filter-rspamd > > > However, I don't see any messages being processed by rspamd. Nor do I see > > > any indication that data is being sent to rspamd (nothing in the logs, no > > > stats appearing in the web UI). > > > > can you show full logs for a sample smtpd session that didn't go through > > rspamd ? > > Is this what you're looking for? > > Aug 21 12:42:22 host smtpd[71198]: 43e03ee20005a41f smtp connected > address=x.x.x.x host=***t.com > Aug 21 12:42:23 host smtpd[71198]: 43e03ee20005a41f smtp message > msgid= size=338369 nrcpt=1 proto=ESMTP > Aug 21 12:42:23 host smtpd[71198]: 43e03ee20005a41f smtp envelope > evpid= > from=> to=<***.***> > Aug 21 12:42:24 host smtpd[71198]: 43e03ee20005a41f smtp disconnected > reason=quit > > The msgid reveals some additional data, but the server doesn't manage final > delivery--emails are received and relayed only. So the additional message > information is related to the outbound (relayed) email but I can provide if > needed. > sorry but this is tricky to troubleshoot with so few logs, obfuscated on top of it :-/ -- Gilles Chehade @poolpOrg https://www.poolp.orgpatreon: https://www.patreon.com/gilles
FLOSS Weekly 543 OpenSMTPD
Hello everyone, I was invited to talk a bit about SMTP and OpenSMTPD in FLOSS Weekly. Here is the link in case you're interested: https://twit.tv/shows/floss-weekly/episodes/543 Cheers -- Gilles Chehade @poolpOrg https://www.poolp.orgpatreon: https://www.patreon.com/gilles
Re: smtpd not passing data to rspamd
‐‐‐ Original Message ‐‐‐ On Wednesday, August 21, 2019 8:28 AM, Gilles Chehade wrote: > On Wed, Aug 21, 2019 at 03:22:39PM +, Thomas Smith wrote: > > > Hi, > > I've setup filter-rspamd with rspamd. Both appear to be running (smtpd and > > rspamd), I'm able to query rspamd's controller, access the web UI; smtpd is > > processing and delivering mail as expected. > > ps wuax | grep rspam > > root 86736 0.0 0.4 45236 4008 ?? I 6:30AM 0:00.05 rspamd: main process > > (rspamd) > > _rspamd 32135 0.0 1.0 45344 10140 ?? S 6:30AM 0:00.23 rspamd: rspamd_proxy > > process (localhost:11332) (rspamd) > > _rspamd 4059 0.0 1.4 45688 14632 ?? S 6:30AM 0:01.63 rspamd: controller > > process (localhost:11334) (rspamd) > > _rspamd 16743 0.0 1.1 45384 11020 ?? S 6:30AM 0:00.33 rspamd: normal > > process (localhost:11333) (rspamd) > > _smtpd 32851 0.0 0.4 105520 3624 ?? I 6:56AM 0:00.01 > > /usr/local/bin/filter-rspamd > > _smtpd 68802 0.0 0.1 844 808 ?? Ip 6:56AM 0:00.00 sh -c > > /usr/local/bin/filter-rspamd > > However, I don't see any messages being processed by rspamd. Nor do I see > > any indication that data is being sent to rspamd (nothing in the logs, no > > stats appearing in the web UI). > > can you show full logs for a sample smtpd session that didn't go through > rspamd ? Is this what you're looking for? Aug 21 12:42:22 host smtpd[71198]: 43e03ee20005a41f smtp connected address=x.x.x.x host=***t.com Aug 21 12:42:23 host smtpd[71198]: 43e03ee20005a41f smtp message msgid= size=338369 nrcpt=1 proto=ESMTP Aug 21 12:42:23 host smtpd[71198]: 43e03ee20005a41f smtp envelope evpid= from=to=<***.***> Aug 21 12:42:24 host smtpd[71198]: 43e03ee20005a41f smtp disconnected reason=quit The msgid reveals some additional data, but the server doesn't manage final delivery--emails are received and relayed only. So the additional message information is related to the outbound (relayed) email but I can provide if needed.
Re: smtpd not passing data to rspamd
On Wed, Aug 21, 2019 at 03:22:39PM +, Thomas Smith wrote: > Hi, > > I've setup filter-rspamd with rspamd. Both appear to be running (smtpd and > rspamd), I'm able to query rspamd's controller, access the web UI; smtpd is > processing and delivering mail as expected. > > ps wuax | grep rspam > root 86736 0.0 0.4 45236 4008 ?? I 6:30AM0:00.05 rspamd: > main process (rspamd) > _rspamd 32135 0.0 1.0 45344 10140 ?? S 6:30AM0:00.23 rspamd: > rspamd_proxy process (localhost:11332) (rspamd) > _rspamd 4059 0.0 1.4 45688 14632 ?? S 6:30AM0:01.63 rspamd: > controller process (localhost:11334) (rspamd) > _rspamd 16743 0.0 1.1 45384 11020 ?? S 6:30AM0:00.33 rspamd: > normal process (localhost:11333) (rspamd) > _smtpd 32851 0.0 0.4 105520 3624 ?? I 6:56AM0:00.01 > /usr/local/bin/filter-rspamd > _smtpd 68802 0.0 0.1 844 808 ?? Ip 6:56AM0:00.00 sh -c > /usr/local/bin/filter-rspamd > > However, I don't see any messages being processed by rspamd. Nor do I see any > indication that data is being sent to rspamd (nothing in the logs, no stats > appearing in the web UI). > can you show full logs for a sample smtpd session that didnt go through rspamd ? > smtpd.conf: > filter "rspamd" proc-exec "/usr/local/bin/filter-rspamd" > listen on egress tls hostname $mx_domain pki $mx_domain filter "rspamd" > > 'smtpd -d -v': > debug: smtp: listen on x.x.x.x port 25 flags 0x2401 pki "" ca "" > > I also don't see any debug messages regarding rspamd. > your config is correct -- Gilles Chehade @poolpOrg https://www.poolp.orgpatreon: https://www.patreon.com/gilles
smtpd not passing data to rspamd
Hi, I've setup filter-rspamd with rspamd. Both appear to be running (smtpd and rspamd), I'm able to query rspamd's controller, access the web UI; smtpd is processing and delivering mail as expected. ps wuax | grep rspam root 86736 0.0 0.4 45236 4008 ?? I 6:30AM0:00.05 rspamd: main process (rspamd) _rspamd 32135 0.0 1.0 45344 10140 ?? S 6:30AM0:00.23 rspamd: rspamd_proxy process (localhost:11332) (rspamd) _rspamd 4059 0.0 1.4 45688 14632 ?? S 6:30AM0:01.63 rspamd: controller process (localhost:11334) (rspamd) _rspamd 16743 0.0 1.1 45384 11020 ?? S 6:30AM0:00.33 rspamd: normal process (localhost:11333) (rspamd) _smtpd 32851 0.0 0.4 105520 3624 ?? I 6:56AM0:00.01 /usr/local/bin/filter-rspamd _smtpd 68802 0.0 0.1 844 808 ?? Ip 6:56AM0:00.00 sh -c /usr/local/bin/filter-rspamd However, I don't see any messages being processed by rspamd. Nor do I see any indication that data is being sent to rspamd (nothing in the logs, no stats appearing in the web UI). My current rspamd configuration is very simple--it's currently minimally configured (following their QuickStart guide), just trying to get the communications working right now. 'uname -a': OpenBSD 6.6 GENERIC#219 amd64 rspamd options changed (local.d): options.inc: local_addrs dns nameserver redis.conf: servers worker-controller.inc: password smtpd.conf: filter "rspamd" proc-exec "/usr/local/bin/filter-rspamd" listen on egress tls hostname $mx_domain pki $mx_domain filter "rspamd" 'smtpd -d -v': debug: smtp: listen on x.x.x.x port 25 flags 0x2401 pki "" ca "" I also don't see any debug messages regarding rspamd.
Re: Question about OpenSMTPD and Debian package and filters/spam filtering
> On 21 Aug 2019, at 13:58, Gilles Chehade wrote: > > On Wed, Aug 21, 2019 at 12:50:10PM +0200, Michiel van Es wrote: >> Hi! >> > > Hi, > > >> I am running a small VPS with 1 GB memory with Debian 10 amd64 with >> OpenSMTPD (6.0.3) for private email and am looking what my best options are >> to limit spam. >> I know there are some filters from Joerg >> (https://www.mail-archive.com/misc@opensmtpd.org/msg04402.html) but am not >> sure if these will work with my version of OpenSMTPD (I get a syntax error >> when trying the old filter syntax). >> >> I can also relay everything to Amavisd/SpamAssassin but then email won???t >> get blocked at the SMTP level, also ASSP or Rspamd is an option but they are >> pretty resource intensive and will eat all my VPS memory ;) >> >> What would be my best option? >> > > 6.0.3 is a fairly old version and there aren't many options available. > > if you're forced to stick with that version, which suffers from at least > one denial of service as far as I know, your best option is to relay via > something like SpamPD so it can interface with SpamAssassin, but this is > not going to operate at SMTP level, it will happen at delivery time. That’s interesting since Debian has a good track record of back porting security fixes in their stable packages. I will ask the maintainer if he applied the patch or upgraded the package to latest version. For now I use spampd which works fine for bayesian spam detection. > > there will be no way of blocking at SMTP level before next release 6.6.0 > that is going to happen in a few weeks, during October, so any option is > going to be post delivery: either as a custom MDA, or as a relay via for > some smtp proxy that will reinject in smtpd like the dkimproxy stuff. I will wait for 6.6.0 ;) > > your best option would really be to build from source 6.4.2: it will not > block at SMTP level but will provide mechanisms to ease interfacing with > spamassassin or rspamd for post-SMTP handling. > > if you're not too easily scared, running the development version is good > too because it's very close to release now, very stable and will not get > much changes until October as I'm busy busy these days ;-) Might give that a try, thanks :) > > >> I like to do some DNSBL and SpamAsssassin checks if possible. >> >> My config if that is to any use to give some insights: >> >> pki server.pragmasec.nl certificate >> "/etc/letsencrypt/live/pragmasec.nl/fullchain.pem" >> pki server.pragmasec.nl key "/etc/letsencrypt/live/pragmasec.nl/privkey.pem" >> listen on localhost >> listen on eth0 port 25 tls pki server.pragmasec.nl hostname >> server.pragmasec.nl auth-optional >> listen on eth0 port 587 tls-require pki server.pragmasec.nl hostname >> server.pragmasec.nl auth >> table vdomains file:/etc/mail/domains >> table vusers file:/etc/mail/vusers >> expire 7d >> limit mta inet4 >> accept from any for domain virtual deliver to mda >> "/usr/lib/dovecot/dovecot-lda -f %{sender} -a %{rcpt}" >> accept from local for any relay >> >> Cheers, >> >> Michiel >> >> >> > > -- > Gilles Chehade @poolpOrg > > https://www.poolp.orgpatreon: https://www.patreon.com/gilles
Re: Question about OpenSMTPD and Debian package and filters/spam filtering
On Wed, Aug 21, 2019 at 12:50:10PM +0200, Michiel van Es wrote: > Hi! > Hi, > I am running a small VPS with 1 GB memory with Debian 10 amd64 with OpenSMTPD > (6.0.3) for private email and am looking what my best options are to limit > spam. > I know there are some filters from Joerg > (https://www.mail-archive.com/misc@opensmtpd.org/msg04402.html) but am not > sure if these will work with my version of OpenSMTPD (I get a syntax error > when trying the old filter syntax). > > I can also relay everything to Amavisd/SpamAssassin but then email won???t > get blocked at the SMTP level, also ASSP or Rspamd is an option but they are > pretty resource intensive and will eat all my VPS memory ;) > > What would be my best option? > 6.0.3 is a fairly old version and there aren't many options available. if you're forced to stick with that version, which suffers from at least one denial of service as far as I know, your best option is to relay via something like SpamPD so it can interface with SpamAssassin, but this is not going to operate at SMTP level, it will happen at delivery time. there will be no way of blocking at SMTP level before next release 6.6.0 that is going to happen in a few weeks, during October, so any option is going to be post delivery: either as a custom MDA, or as a relay via for some smtp proxy that will reinject in smtpd like the dkimproxy stuff. your best option would really be to build from source 6.4.2: it will not block at SMTP level but will provide mechanisms to ease interfacing with spamassassin or rspamd for post-SMTP handling. if you're not too easily scared, running the development version is good too because it's very close to release now, very stable and will not get much changes until October as I'm busy busy these days ;-) > I like to do some DNSBL and SpamAsssassin checks if possible. > > My config if that is to any use to give some insights: > > pki server.pragmasec.nl certificate > "/etc/letsencrypt/live/pragmasec.nl/fullchain.pem" > pki server.pragmasec.nl key "/etc/letsencrypt/live/pragmasec.nl/privkey.pem" > listen on localhost > listen on eth0 port 25 tls pki server.pragmasec.nl hostname > server.pragmasec.nl auth-optional > listen on eth0 port 587 tls-require pki server.pragmasec.nl hostname > server.pragmasec.nl auth > table vdomains file:/etc/mail/domains > table vusers file:/etc/mail/vusers > expire 7d > limit mta inet4 > accept from any for domain virtual deliver to mda > "/usr/lib/dovecot/dovecot-lda -f %{sender} -a %{rcpt}" > accept from local for any relay > > Cheers, > > Michiel > > > -- Gilles Chehade @poolpOrg https://www.poolp.orgpatreon: https://www.patreon.com/gilles
Question about OpenSMTPD and Debian package and filters/spam filtering
Hi! I am running a small VPS with 1 GB memory with Debian 10 amd64 with OpenSMTPD (6.0.3) for private email and am looking what my best options are to limit spam. I know there are some filters from Joerg (https://www.mail-archive.com/misc@opensmtpd.org/msg04402.html) but am not sure if these will work with my version of OpenSMTPD (I get a syntax error when trying the old filter syntax). I can also relay everything to Amavisd/SpamAssassin but then email won’t get blocked at the SMTP level, also ASSP or Rspamd is an option but they are pretty resource intensive and will eat all my VPS memory ;) What would be my best option? I like to do some DNSBL and SpamAsssassin checks if possible. My config if that is to any use to give some insights: pki server.pragmasec.nl certificate "/etc/letsencrypt/live/pragmasec.nl/fullchain.pem" pki server.pragmasec.nl key "/etc/letsencrypt/live/pragmasec.nl/privkey.pem" listen on localhost listen on eth0 port 25 tls pki server.pragmasec.nl hostname server.pragmasec.nl auth-optional listen on eth0 port 587 tls-require pki server.pragmasec.nl hostname server.pragmasec.nl auth table vdomains file:/etc/mail/domains table vusers file:/etc/mail/vusers expire 7d limit mta inet4 accept from any for domain virtual deliver to mda "/usr/lib/dovecot/dovecot-lda -f %{sender} -a %{rcpt}" accept from local for any relay Cheers, Michiel
Re: forcing SMTP authentication
> That last rule is essentially "accept from any for (pretty much) any" so > you have created an open relay. > > Replace the "from any" with "from local" so the rule reads as: > >accept from local for ! domain 486.hu relay via > tls+auth://t-onl...@mail.t-online.hu auth > > This should be much better. Thanks, this did the trick! :) I thought when it comes to relaying, "auth-optional" ensures that authentication is already done and the relaying rule is processed according to this. Regards,
Re: forcing SMTP authentication
On Wed, Aug 21, 2019 at 07:39:42AM +0200, Selmeci Tam?s wrote: > Hello! > > In brief: STARTTLS is enabled, there is a self-signed certificate for > encryption (better than nothing), smarthost is used to send mails from > my domain. My problem is that it still accepts SMTP connections (over > TLS) without authentication. What I want: > - anybody can send email to my email address in my domain (now it's > working); > - relaying through my SMTP server is allowed only after successful > authentication (now anybody can relay through my server without > authentication, e.g. to send spams). Authentication should be based on > regular /etc/passwd file (local users of the computer). In order to > hide the passwords, STARTTLS should be used; > > It's a rather simple configuration, but I wasn't able to set it up. If > I put 'auth' into the 'listen on' line, it needs authentication to any > access of the SMTP server, so other machines (e.g. from google.com) > can't send me mails. Using 'authenticated' in 'accept from' directives > also didn't do the trick appropriately (it wasn't able to receive any > mails at all). > > Could you please help me out with this? > > Thanks, regards, > --- > --- > pki mail.486.hu certificate "/etc/smtpd/mail.486.hu.crt" > pki mail.486.hu key "/etc/smtpd/mail.486.hu.key" > > table cred file:/etc/smtpd/cred > > listen on eth0 port 25 hostname mail.486.hu tls-require > listen on localhost port 25 hostname mail.486.hu tls-require > you should add: listen on eth0 port 587 hostname mail.486.hu tls-require auth > # Storing mails arriving at the domain '486.hu'. > accept from any for domain 486.hu deliver to mbox > > # If the recipient is out of domain '486.hu', the mail is relayed through the > # smarthost using TLS and authentication, see 'cred' file. > accept from any for ! domain 486.hu relay via > tls+auth://t-onl...@mail.t-online.hu auth > That last rule is essentially "accept from any for (pretty much) any" so you have created an open relay. Replace the "from any" with "from local" so the rule reads as: accept from local for ! domain 486.hu relay via tls+auth://t-onl...@mail.t-online.hu auth This should be much better. -- Gilles Chehade @poolpOrg https://www.poolp.orgpatreon: https://www.patreon.com/gilles
Re: forcing SMTP authentication
On Wed, 21 Aug 2019 06:50:05 + Lévai, Dániel wrote: > No it doesn't, that's the whole point... Very strange. Currently I'm in the office, the mail server is at home. I tried with two mail clients (sylpheed, Evolution) with a fake account to use the mail server without authentication to send email to my gmail address - at it worked all the time. Maybe something went wrong during ./configure? The /var/log/messages logs are attached in a file. Regards, -- Selmeci Tamás Aug 21 09:11:37 486 mail.info smtpd[13132]: 242a473f710cb686 smtp event=connected address=217.150.134.30 host=217.150.134.30 Aug 21 09:11:38 486 mail.info smtpd[13132]: 242a473f710cb686 smtp event=starttls address=217.150.134.30 host=217.150.134.30 ciphers="version=TLSv1.2, cipher=ECDHE-RSA-AES256-GCM-SHA384, bits=256" Aug 21 09:11:38 486 mail.info smtpd[13132]: 242a473f710cb686 smtp event=message address=217.150.134.30 host=217.150.134.30 msgid=4ece59a6 from= to= size=502 ndest=1 proto=ESMTP Aug 21 09:11:38 486 mail.info smtpd[13132]: 242a4742bfc88a7f mta event=connecting address=tls://84.2.46.3:25 host=mail.t-online.hu Aug 21 09:11:38 486 mail.info smtpd[13132]: 242a473f710cb686 smtp event=closed address=217.150.134.30 host=217.150.134.30 reason=quit Aug 21 09:11:38 486 mail.info smtpd[13132]: 242a4742bfc88a7f mta event=connected Aug 21 09:11:38 486 mail.info smtpd[13132]: 242a4742bfc88a7f mta event=starttls ciphers=version=TLSv1.2, cipher=ECDHE-RSA-AES256-GCM-SHA384, bits=256 Aug 21 09:11:38 486 mail.err smtpd[13131]: warn: unable to load CA file /etc/ssl/cert.pem: No such file or directory Aug 21 09:11:38 486 mail.info smtpd[13132]: smtp-out: Server certificate verification failed on session 242a4742bfc88a7f Aug 21 09:11:39 486 mail.info smtpd[13132]: 242a4742bfc88a7f mta event=delivery evpid=4ece59a66756afe7 from= to= rcpt=<-> source="192.168.1.153" relay="84.2.46.3 (mail.t-online.hu)" delay=1s result="Ok" stat="250 2.0.0 Ok: queued as Aug 21 09:11:42 486 mail.info smtpd[13132]: 242a474391a6416d smtp event=connected address=209.85.210.43 host=mail-ot1-f43.google.com Aug 21 09:11:43 486 mail.info smtpd[13132]: 242a474391a6416d smtp event=starttls address=209.85.210.43 host=mail-ot1-f43.google.com ciphers="version=TLSv1.2, cipher=ECDHE-RSA-AES256-GCM-SHA384, bits=256"
Re: forcing SMTP authentication
On 8/21/19 8:47 AM, Selmeci Tamás wrote: > On Wed, 21 Aug 2019 08:19:24 +0200 Martijn van Duren > wrote: > >> From smtpd.conf(5): >> >> auth-optional [] >> Support SMTPAUTH optionally: clients need not >> authenticate, but may do so. This allows a listen on >> directive to both accept incoming mail from untrusted >> senders and permit outgoing mail from authenticated >> users >> (using match auth). It can be used in situations where >> it is not possible to listen on a separate port (usually >> the submission port, 587) for users to authenticate. > > Sounds good, but unauthenticated relaying still works with this... > auth-optional [] ...snip... (using match auth) ...snip... match options action name If at least one mail envelope matches the options of one match action directive, receive the incoming message, put a copy into each matching envelope, and atomically save the envelopes to the mail spool for later processing by the respective dispatcher name. ...snip... [!] auth Matches transactions which have been authenticated.