Support for XCLIENT STRESS=yes/no
Hello, i'm using a lot of automated tests to test my postfix configuration. The XCLIENT command is very handy for such checks. I thought about adding a STRESS parameter to the command to be able to also write test plans to test under-stress-behavior. It was easy to add the parameters to the XCLIENT command, but i seems like postfix pre-parses almost every parameter. And changing the stress option at this stage is too late. For example, i could get the banner working by replacing the value with a call to mail_conf lookup: // smtpd_chat_reply(state, 220 %s, var_smtpd_banner); smtpd_chat_reply(state, 220 %s, mail_conf_lookup_eval(VAR_SMTPD_BANNER)); # telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.bla.de mail server (idle) XCLIENT STRESS=yes 220 mail.bla.de mail server (stress) But to be usefull at least the restriction maps/classes must be reloaded. I was unable to find an easy way to achieve this at the XCLIENT state. Any suggestions? Markus
tls_policy_map, combination with transport_maps
Hello!
*main.cf config:*
smtp_tls_policy_maps = hash:/etc/postfix/maps/tls_policy
transport_maps = proxy:mysql:/etc/postfix/mysql/relay-transport.cf
*Postmap query:*
postmap -q i...@testkund.domain.tld
mysql:/etc/postfix/mysql/relay-transport.cf
smtp:d748.dev-cust.domain.tld:
postmap -q [d748.dev-cust.domain.tld]: hash:/etc/postfix/maps/tls_policy
[d748.dev-cust.domain.tld]: none
/The TLS policy table is indexed by the full next-hop destination,
which is either the recipient domain, or the verbatim next-hop specified
in the transport table, $local_transport, $virtual_transport,
$relay_transport or $default_transport. This includes any enclosing
square brackets and any non-default destination server port suffix. The
LMTP socket type prefix (inet: or unix:) is not included in the lookup
key./
*
**default setting is:*
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt
smtp_tls_loglevel = 1
smtp_tls_security_level = may
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtp_tls_protocols = !SSLv2,!TLSv1.1,!TLSv1.2
smtp_tls_exclude_ciphers = 3DES:MD5
smtp_tls_policy_maps = hash:/etc/postfix/maps/tls_policy
It isn't using the tls_policy_maps setting. I've must have done
something wrong or totally misunderstood this.
This isn’t working as-well: testkund.domain.tld none (in tls_policy_maps)
postfix 2.9.6-2 amd64 (on debian wheezy) (tested
2.11.1 as-well)
Mailflow:
ORGINATING-SMTPD-AFTER-QUEUE-Amavisd-new-SMTPD:10030-SMTP-END (all
in some instance( /etc/postfix))
And between (SMTP-END) I want to use the tls_policy_map.
I hope I’ve described the problem so you understands me :)
PS. no chroot what-so ever.
signature.asc
Description: OpenPGP digital signature
Re: Idea: multiple actions in access/header_checks/policy results
Am 22.10.2014 um 15:49 schrieb Wietse Venema:
The basic idea is to permit more than one action in an access(5) or
header_checks(5) table lookup result or SMTPD policy response:
A few examples to clarify:
/etc/postfix/access:
# The new multi-action form.
1.2.3.4 { prepend header text... } other actions...
/etc/postfix/header_checks:
# prepend+ignore should be equivalent to replace.
/foo/ { prepend replacement header text... } ignore
policy service:
# The new multi-action form.
action = { prepend new header text... } other actions...
In all cases the historical one-action form still works as before.
There is a small gotcha: the $number expansion in regexp and pcre
table lookup results happens before {} parsing. This would make {}
parsing dependent on the content of $number expansion, i.e. untrusted
data, creating a command injection vulnerability.
To avoid this:
- Either the {} parsing needs to be done before $number expansion.
I'm rejecting this option because it is likely to break the
orthogonality of the Postfix table lookup interface, and that must
be avoided at all cost.
- Or the $number expansion stays where it is now, before {} parsing,
but the caller can specify a filter that censors out {} in $number
expansions, so that the caller's command parser cannot be tricked
into executing the wrong command. This extends the table API with
one function that does nothing except for regexp and pcre tables.
(Of course nothing in Postfix can protect against accidents caused
by text manipulations inside a policy daemon; they would have to
censor themselves, or Postfix would need an option to disable {}
support for policy filter replies).
The second option would be applicable for any feature that uses
table lookups with untrusted data in the lookup key and that does
non-trivial parsing of the lookup result. Examples of such features
are header/body_checks, smtp_header/body_checks, milter_header_checks,
access maps, but not smtpd_command_filter or *_delivery_status_filter.
Wietse
Re: Support for XCLIENT STRESS=yes/no
Markus Benning: Hello, i'm using a lot of automated tests to test my postfix configuration. The XCLIENT command is very handy for such checks. I thought about adding a STRESS parameter to the command to be able to also write test plans to test under-stress-behavior. It cannot work. Postfix configuration parameters are expanded BEFORE Postfix initializes the SMTP protocol engine. There is an obvious chicken-and-egg problem if it were to do this the other way around. Wietse It was easy to add the parameters to the XCLIENT command, but i seems like postfix pre-parses almost every parameter. And changing the stress option at this stage is too late. For example, i could get the banner working by replacing the value with a call to mail_conf lookup: // smtpd_chat_reply(state, 220 %s, var_smtpd_banner); smtpd_chat_reply(state, 220 %s, mail_conf_lookup_eval(VAR_SMTPD_BANNER)); # telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.bla.de mail server (idle) XCLIENT STRESS=yes 220 mail.bla.de mail server (stress) But to be usefull at least the restriction maps/classes must be reloaded. I was unable to find an easy way to achieve this at the XCLIENT state. Any suggestions? Markus
Re: tls_policy_map, combination with transport_maps
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 10/23/2014 8:32 AM, Patrik Båt wrote:
Hello!
*main.cf config:* smtp_tls_policy_maps =
hash:/etc/postfix/maps/tls_policy transport_maps =
proxy:mysql:/etc/postfix/mysql/relay-transport.cf
*Postmap query:* postmap -q i...@testkund.domain.tld
mysql:/etc/postfix/mysql/relay-transport.cf
smtp:d748.dev-cust.domain.tld:
No brackets.
postmap -q [d748.dev-cust.domain.tld]:
hash:/etc/postfix/maps/tls_policy
[d748.dev-cust.domain.tld]: none
brackets.
The entries don't match.
-- Noel Jones
/The TLS policy table is indexed by the full next-hop
destination, which is either the recipient domain, or the
verbatim next-hop specified in the transport table,
$local_transport, $virtual_transport, $relay_transport or
$default_transport. This includes any enclosing square brackets
and any non-default destination server port suffix. The LMTP
socket type prefix (inet: or unix:) is not included in the
lookup key./ * **default setting is:* smtp_tls_CAfile =
/etc/ssl/certs/ca-certificates.crt smtp_tls_loglevel = 1
smtp_tls_security_level = may smtp_tls_session_cache_database =
btree:${data_directory}/smtp_scache smtp_tls_protocols =
!SSLv2,!TLSv1.1,!TLSv1.2 smtp_tls_exclude_ciphers = 3DES:MD5
smtp_tls_policy_maps = hash:/etc/postfix/maps/tls_policy
It isn't using the tls_policy_maps setting. I've must have
done something wrong or totally misunderstood this.
This isn’t working as-well: testkund.domain.tld none (in
tls_policy_maps)
postfix 2.9.6-2 amd64 (on debian wheezy)
(tested 2.11.1 as-well)
Mailflow:
ORGINATING-SMTPD-AFTER-QUEUE-Amavisd-new-SMTPD:10030-SMTP-END
(all in some instance( /etc/postfix))
And between (SMTP-END) I want to use the tls_policy_map.
I hope I’ve described the problem so you understands me :)
PS. no chroot what-so ever.
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (MingW32)
iQEcBAEBAgAGBQJUSRBQAAoJEJGRUHb5Oh6gAzAIAIhVX40/yufZAShzQXO1hI15
eYaz3oXqEcVPTO/JojQGeXgZHsCwlmeyNOX4e8qtYtx0rHXpBhe7Vl52yDGPSOyU
/2ARYgiy128ycLW7UEeVCS7shdPaeJtv3S3EJ/FZzbJWL/tzruneiOr+QCmLAzJY
il8cznI+Mm6TApVK+F/6FN5gYeYohY6fIvLs2AMelmBCC9cu6si2Kr9N2dvtK5hH
TdWJjO0qBP0X7VUy9bkLG0tkDkf1hcZq9pjsHK3P80PQSiuIAYwKuJKmwiQ30uZH
VWgoKXZefEskylwdRkpitLt3pX4dT6gBvpr+QDvbJ00iYyFCH6JRI27i9n5IJgw=
=seIX
-END PGP SIGNATURE-
Re: Idea: multiple actions in access/header_checks/policy results
On Thu, Oct 23, 2014 at 09:39:24AM -0400, Wietse Venema wrote:
In all cases the historical one-action form still works as before.
There is a small gotcha: the $number expansion in regexp and pcre
table lookup results happens before {} parsing. This would make {}
parsing dependent on the content of $number expansion, i.e. untrusted
data, creating a command injection vulnerability.
I noticed the possibility when you sent the original message.
- Or the $number expansion stays where it is now, before {} parsing,
but the caller can specify a filter that censors out {} in $number
expansions, so that the caller's command parser cannot be tricked
into executing the wrong command. This extends the table API with
one function that does nothing except for regexp and pcre tables.
It might alternatively make sense to allow escaping via \{, \}
and \\, and to signal the pcre/regexp drivers to apply such
escapes to interpolated strings.
(Of course nothing in Postfix can protect against accidents caused
by text manipulations inside a policy daemon; they would have to
censor themselves, or Postfix would need an option to disable {}
support for policy filter replies).
As for policy tables, it might be better to support multiple action
values in a single reply, though I understand that this may be
difficult to support in the current attr_client API, can
ATTR_TYPE_FUNC deal with repeated instances of the same attribute?
--
Viktor.
warning: psc_dnsbl_request: connect to private/dnsblog service: Resource temporarily unavailable
Hi
I'm seeing multple error messages like this in my Postfix configuration:
warning: psc_dnsbl_request: connect to private/dnsblog service: Resource
temporarily unavailable
I also see from debugging with the -v command that not all blacklists are
checked.
I have found the error comes from this part of postscreen_dnsbl.c:
if ((fd = LOCAL_CONNECT(psc_dnsbl_service, NON_BLOCKING, 1))
0) {
msg_warn(%s: connect to %s service: %m,
myname, psc_dnsbl_service);
continue;
}
Appeantly the LOCAL_CONNECT is using unix_connect.c and opens a socket
connection.
So why do I get those messages and can't I open a socket connection to
private/dnsblog?
# cat /proc/sys/fs/file-max
203818
# postconf -d | grep mail_version
mail_version = 2.9.6
Kind regards
Laurens Van Acker
Re: Idea: multiple actions in access/header_checks/policy results
Viktor Dukhovni:
It might alternatively make sense to allow escaping via \{, \}
and \\, and to signal the pcre/regexp drivers to apply such
escapes to interpolated strings.
Postfix does not support backslash escapes. Introducing that now
would be too disruptive. Introducing them only for special cases
would cause more problems than they solve. I fight against special
cases and for orthogonality.
(Of course nothing in Postfix can protect against accidents caused
by text manipulations inside a policy daemon; they would have to
censor themselves, or Postfix would need an option to disable {}
support for policy filter replies).
As for policy tables, it might be better to support multiple action
values in a single reply, though I understand that this may be
difficult to support in the current attr_client API, can
ATTR_TYPE_FUNC deal with repeated instances of the same attribute?
Sorry, it must be the same interface as access tables. Special
cases are taboo.
Wietse
Re: warning: psc_dnsbl_request: connect to private/dnsblog service: Resource temporarily unavailable
Laurens Van Acker: warning: psc_dnsbl_request: connect to private/dnsblog service: Resource temporarily unavailable Tune up your kernel. Wietse
RE: warning: psc_dnsbl_request: connect to private/dnsblog service: Resource temporarily unavailable
Hi Wietse I have 3.15.4-x86_64-linode45, the latest stable form Linode, from 2014-07-08, which one is the minimum? Kind regards Laurens Van Acker -Original Message- From: owner-postfix-us...@postfix.org [mailto:owner-postfix-us...@postfix.org] On Behalf Of Wietse Venema Sent: donderdag 23 oktober 2014 17:08 To: Laurens Van Acker Cc: postfix-users@postfix.org Subject: Re: warning: psc_dnsbl_request: connect to private/dnsblog service: Resource temporarily unavailable Laurens Van Acker: warning: psc_dnsbl_request: connect to private/dnsblog service: Resource temporarily unavailable Tune up your kernel. Wietse
Re: warning: psc_dnsbl_request: connect to private/dnsblog service: Resource temporarily unavailable
On Thu, Oct 23, 2014 at 05:02:32PM +0200, Laurens Van Acker wrote: warning: psc_dnsbl_request: connect to private/dnsblog service: Resource temporarily unavailable A backlog of connections to the service? Ordinarily master will spawn more instances of the service if all instances are busy. Report the master.cf entry for dnsblog, and the number of processes currently running. You may should be able to get information about pending unaccepted connections out of the kernel. -- Viktor.
Re: Idea: multiple actions in access/header_checks/policy results
On Thu, Oct 23, 2014 at 11:07:30AM -0400, Wietse Venema wrote:
Viktor Dukhovni:
It might alternatively make sense to allow escaping via \{, \}
and \\, and to signal the pcre/regexp drivers to apply such
escapes to interpolated strings.
Postfix does not support backslash escapes. Introducing that now
would be too disruptive. Introducing them only for special cases
would cause more problems than they solve. I fight against special
cases and for orthogonality.
(Of course nothing in Postfix can protect against accidents caused
by text manipulations inside a policy daemon; they would have to
censor themselves, or Postfix would need an option to disable {}
support for policy filter replies).
As for policy tables, it might be better to support multiple action
values in a single reply, though I understand that this may be
difficult to support in the current attr_client API, can
ATTR_TYPE_FUNC deal with repeated instances of the same attribute?
Sorry, it must be the same interface as access tables. Special
cases are taboo.
I'm not fond of lossful mangling of the input, especially if policy
services are liable to get it wrong. Perhaps this idea needs to
wait until the Postfix dictionary layer understands multiple results,
returning a list rather than a concatenation. At that point,
suitable variants of pcre/regexp or directives at the top of the
file can turn on list splitting of the RHS prior to sub-pattern
replacement.
--
Viktor.
RE: warning: psc_dnsbl_request: connect to private/dnsblog service: Resource temporarily unavailable
Hi Viktor This is my master.cf: dnsblog unix - - n - 0 dnsblog Number of processes: # ps -Al | wc -l 159 How do I get the number of pending unaccepted connections? It's only one connection, for example: Oct 23 17:15:54 uk01 postfix/anvil[16202]: statistics: max connection rate 2/60s for (smtpd:5.134.1.44) at Oct 23 17:07:01 Oct 23 17:15:54 uk01 postfix/anvil[16202]: statistics: max connection count 1 for (smtpd:192.64.236.99) at Oct 23 17:06:25 Oct 23 17:15:54 uk01 postfix/anvil[16202]: statistics: max cache size 4 at Oct 23 17:08:56 Oct 23 17:17:01 uk01 postfix/postscreen[16541]: CONNECT from [5.134.1.44]:48628 to [109.74.192.6]:25 Oct 23 17:17:01 uk01 postfix/postscreen[16541]: warning: psc_dnsbl_request: connect to private/dnsblog service: Resource temporarily unavailable Oct 23 17:17:01 uk01 postfix/postscreen[16541]: warning: psc_dnsbl_request: connect to private/dnsblog service: Resource temporarily unavailable Oct 23 17:17:01 uk01 postfix/postscreen[16541]: CONNECT from [5.134.1.44]:48629 to [109.74.192.6]:25 Oct 23 17:17:03 uk01 postfix/postscreen[16541]: PASS OLD [5.134.1.44]:48628 Oct 23 17:17:03 uk01 postfix/postscreen[16541]: PASS OLD [5.134.1.44]:48629 Oct 23 17:17:03 uk01 postfix/smtpd[16542]: connect from s02.go-media.be[5.134.1.44] Oct 23 17:17:03 uk01 postfix/smtpd[16568]: connect from s02.go-media.be[5.134.1.44] Oct 23 17:17:03 uk01 postfix/smtpd[16542]: 9CC0F69: client=s02.go-media.be[5.134.1.44] Oct 23 17:17:03 uk01 postfix/cleanup[16460]: 9CC0F69: message-id=1414077018077.549543...@mg-10.p6.xq Oct 23 17:17:03 uk01 postfix/smtpd[16568]: A14A790: client=s02.go-media.be[5.134.1.44] Oct 23 17:17:03 uk01 postfix/cleanup[16571]: A14A790: message-id=20141023151141.904EC31A@evilapp-004 Oct 23 17:17:03 uk01 postfix/qmgr[16147]: A14A790: from=bounce-gfde9e56307ce75151615bada416d131807451dd...@is.msdp1.com, size=10466, nrcpt=1 (queue active) Oct 23 17:17:03 uk01 postfix/smtpd[16568]: disconnect from s02.go-media.be[5.134.1.44] Oct 23 17:17:03 uk01 postfix/qmgr[16147]: 9CC0F69: from=filtered e-mail, size=45373, nrcpt=1 (queue active) Oct 23 17:17:03 uk01 postfix/smtpd[16542]: disconnect from s02.go-media.be[5.134.1.44] Laurens -Original Message- From: owner-postfix-us...@postfix.org [mailto:owner-postfix-us...@postfix.org] On Behalf Of Viktor Dukhovni Sent: donderdag 23 oktober 2014 17:20 To: postfix-users@postfix.org Subject: Re: warning: psc_dnsbl_request: connect to private/dnsblog service: Resource temporarily unavailable On Thu, Oct 23, 2014 at 05:02:32PM +0200, Laurens Van Acker wrote: warning: psc_dnsbl_request: connect to private/dnsblog service: Resource temporarily unavailable A backlog of connections to the service? Ordinarily master will spawn more instances of the service if all instances are busy. Report the master.cf entry for dnsblog, and the number of processes currently running. You may should be able to get information about pending unaccepted connections out of the kernel. -- Viktor.
Re: Support for XCLIENT STRESS=yes/no
Am Donnerstag, den 23.10.2014, 09:42 -0400 schrieb Wietse Venema: Markus Benning: Hello, i'm using a lot of automated tests to test my postfix configuration. The XCLIENT command is very handy for such checks. I thought about adding a STRESS parameter to the command to be able to also write test plans to test under-stress-behavior. It cannot work. Postfix configuration parameters are expanded BEFORE Postfix initializes the SMTP protocol engine. There is an obvious chicken-and-egg problem if it were to do this the other way around. Wietse I already guessed that this may be the case. But anyways, thank you for your advise. I'll look into this again tomorrow. May be I can just add a seperate smtpd process with hardcoded stress=yes on a private port. Markus
Re: Idea: multiple actions in access/header_checks/policy results
Viktor Dukhovni: Sorry, it must be the same interface as access tables. Special cases are taboo. I'm not fond of lossful mangling of the input, especially if policy services are liable to get it wrong. Perhaps this idea needs to wait until the Postfix dictionary layer understands multiple results, returning a list rather than a concatenation. At that point, suitable variants of pcre/regexp or directives at the top of the file can turn on list splitting of the RHS prior to sub-pattern replacement. Postfix has a simple string interface for its key-value store. I have an aversion against changing that into some binary protocol with sub-record structure. Similar to my aversion against systemd logging, among other things. Wietse
reject attachments for certain recipients
Hi, Is there a simple way in Postfix to reject attachments, but only for certain email addresses? Thanks. attachment: andre.vcf
Re: reject attachments for certain recipients
Am 23.10.2014 um 17:47 schrieb Andre Rodier: Is there a simple way in Postfix to reject attachments, but only for certain email addresses? no - not without policy daemons the capabilities of postfix are restricted by intention to keep it safe, stable and maintainable but they are powerful to extend
Postfix/milter benchmarking
Dear Postfix community, I'm developing a new milter and I need to benchmark it when plugged into Postfix. What are my best options for generating an SMTP stream of messages directed at a single Postfix instance at rates on the order of hundreds per second, and measuring SMTP acceptance rates and SMTP response latency? I can run processes on multiple machines if necessary, but the more efficient the better. I saw the smtp-source tool http://www.postfix.org/smtp-source.1.html, but I'm not sure I can tune it to certain output rates and make it take acceptance rate and latency measurements. Are there any other tools people use to benchmark their Postfix setups or, more specifically, milters? Thanks, -Julian
RE: warning: psc_dnsbl_request: connect to private/dnsblog service: Resource temporarily unavailable
Hi Viktor None: # pgrep -x dnsblog | wc -l 0 Not much: # netstat -anp --unix | wc -l 249 The lock file is there: # ls /var/spool/postfix/pid/ inet.26 inet.smtpmaster.pid unix.bounce unix.defer unix.dspam unix.flush unix.maildrop unix.relay unix.showq unix.trace inet.587 inet.submission pass.smtpd unix.cleanup unix.dnsblog unix.dspam-retrain unix.local unix.msn unix.retry unix.smtp # netstat -anp --unix | grep dnsblog unix 2 [ ACC ] STREAM LISTENING 526425 10764/master private/dnsblog unix 2 [ ] DGRAM717894 17046/dnsblog unix 2 [ ] DGRAM716334 17041/dnsblog unix 2 [ ] DGRAM717846 17038/dnsblog unix 2 [ ] DGRAM716310 17037/dnsblog unix 2 [ ] DGRAM717882 17044/dnsblog unix 2 [ ] DGRAM716322 17039/dnsblog unix 2 [ ] DGRAM716346 17043/dnsblog unix 2 [ ] DGRAM717858 17040/dnsblog unix 2 [ ] DGRAM717870 17042/dnsblog unix 2 [ ] DGRAM716360 17045/dnsblog unix 2 [ ] DGRAM716520 17053/dnsblog Met vriendelijke groeten Laurens Van Acker laur...@van.acker.be http://laurens.tel -Original Message- From: owner-postfix-us...@postfix.org [mailto:owner-postfix-us...@postfix.org] On Behalf Of Viktor Dukhovni Sent: donderdag 23 oktober 2014 17:38 To: postfix-users@postfix.org Subject: Re: warning: psc_dnsbl_request: connect to private/dnsblog service: Resource temporarily unavailable On Thu, Oct 23, 2014 at 05:28:21PM +0200, Laurens Van Acker wrote: Hi Viktor This is my master.cf: dnsblog unix - - n - 0 dnsblog Number of processes: # ps -Al | wc -l 159 That's all processes, how about dnsblog processes? On linux: pgrep -x dnsblog | wc -l How do I get the number of pending unaccepted connections? Depends on your OS. # netstat -anp --unix might work on Linux when run as root. You'll want to look at things trying to connect to /var/spool/postfix/private/dnsblog Oct 23 17:15:54 uk01 postfix/anvil[16202]: statistics: max connection rate 2/60s for (smtpd:5.134.1.44) at Oct 23 17:07:01 Oct 23 17:15:54 uk01 postfix/anvil[16202]: statistics: max connection count 1 for (smtpd:192.64.236.99) at Oct 23 17:06:25 Oct 23 17:15:54 uk01 postfix/anvil[16202]: statistics: max cache size 4 at Oct 23 17:08:56 Oct 23 17:17:01 uk01 postfix/postscreen[16541]: CONNECT from [5.134.1.44]:48628 to [109.74.192.6]:25 Oct 23 17:17:01 uk01 postfix/postscreen[16541]: warning: psc_dnsbl_request: connect to private/dnsblog service: Resource temporarily unavailable The process that is supposed to be accepting connections on this socket is stuck. Possibly a kernel bug. On a busy system, there should be one dbsblog process holding an exclusive lock on: /var/spool/postfix/pid/unix.dbsblog this process should be blocked in an accept(2) system call. If clients are trying to connect, but that process remains blocked, your kernel is buggy. -- Viktor.
Re: Postfix/milter benchmarking
Julian Mehnle: Dear Postfix community, I'm developing a new milter and I need to benchmark it when plugged into Postfix. What are my best options for generating an SMTP stream of messages directed at a single Postfix instance at rates on the order of hundreds per second, and measuring SMTP acceptance rates and SMTP response latency? I can run processes on multiple machines if necessary, but the more efficient the better. I saw the smtp-source tool http://www.postfix.org/smtp-source.1.html, but I'm not sure I can tune it to certain output rates and make it take acceptance rate and latency measurements. Try it and see how fast it goes with and without your Milter. Are there any other tools people use to benchmark their Postfix setups or, more specifically, milters? Have you tried smtp benchmark as a search engine query? I have not used any of the programs out there. Wietse
Re: warning: psc_dnsbl_request: connect to private/dnsblog service: Resource temporarily unavailable
On Thu, Oct 23, 2014 at 05:53:01PM +0200, Laurens Van Acker wrote: # pgrep -x dnsblog | wc -l 0 Seems unlikely, below nestat reports dnsblog processes The lock file is there: # ls /var/spool/postfix/pid/ ... unix.dnsblog ... Which process has it locked? # netstat -anp --unix | grep dnsblog unix 2 [ ACC ] STREAM LISTENING 526425 10764/master private/dnsblog unix 2 [ ] DGRAM717894 17046/dnsblog unix 2 [ ] DGRAM716334 17041/dnsblog unix 2 [ ] DGRAM717846 17038/dnsblog unix 2 [ ] DGRAM716310 17037/dnsblog unix 2 [ ] DGRAM717882 17044/dnsblog unix 2 [ ] DGRAM716322 17039/dnsblog unix 2 [ ] DGRAM716346 17043/dnsblog unix 2 [ ] DGRAM717858 17040/dnsblog unix 2 [ ] DGRAM717870 17042/dnsblog unix 2 [ ] DGRAM716360 17045/dnsblog unix 2 [ ] DGRAM716520 17053/dnsblog She DGRAM socket is likely the syslog socket. I'm not seeing any dnsblog processes listening on the actual dnsblog socket. You have to dig deeper, but may lack the skill to do so... I can't spoon-feed you through this one step at a time. :-( -- Viktor.
Re: warning: psc_dnsbl_request: connect to private/dnsblog service: Resource temporarily unavailable
On Thu, Oct 23, 2014 at 04:06:55PM +, Viktor Dukhovni wrote: She DGRAM socket is likely the syslog socket. I'm not seeing any dnsblog processes listening on the actual dnsblog socket. You have to dig deeper, but may lack the skill to do so... I can't spoon-feed you through this one step at a time. :-( Restarting Postfix or rebooting the machine may sweep the problem under the rug. Perhaps your syslog service is congested, if so, make sure you're not doing synchronous logging. -- Viktor.
RE: warning: psc_dnsbl_request: connect to private/dnsblog service: Resource temporarily unavailable
Hi Victor So I have 10 sockets listening, but I have more lists to check. Does that can be a reason? I have also seen that if I remove some lists I get less of those errors and if I add some I have more. Can I higher the number of sockets listening for requests? My main: postscreen_dnsbl_sites = zen.spamhaus.org*2 cbl.abuseat.org*2 b.barracudacentral.org*2 dyna.spamrats.com bl.spameatingmonkey.net bl.spamcop.net free.v4BL.org*2 dnsbl-1.uceprotect.net dnsbl.sorbs.net all.s5h.net swl.spamhaus.org*-4 whitelist.surriel.com*-4 wl.mailspike.net=127.0.0.[17;18]*-3 wl.mailspike.net=127.0.0.[19;20]*-6 So as you can see it are 13 lists and I only have 10 sockets listening. # pgrep -x dnsblog | wc -l Sometimes it gives me 0 and sometimes 10. I guess it depends on if there are mails coming in or not. The locks seem to be owned by dnsblog itself: # lsof /var/spool/postfix/pid/unix.dnsblog # # fuser /var/spool/postfix/pid/unix.dnsblog /var/spool/postfix/pid/unix.dnsblog: 17452 17453 17454 17455 17456 17457 17458 17459 17460 17461 # # ps 17452 PID TTY STAT TIME COMMAND 17452 ?S 0:00 dnsblog -z -t unix -u # ps 17453 PID TTY STAT TIME COMMAND 17453 ?S 0:00 dnsblog -z -t unix -u # # ps 17455 PID TTY STAT TIME COMMAND 17455 ?S 0:00 dnsblog -z -t unix -u # Kind regards Laurens -Original Message- From: owner-postfix-us...@postfix.org [mailto:owner-postfix-us...@postfix.org] On Behalf Of Viktor Dukhovni Sent: donderdag 23 oktober 2014 18:07 To: postfix-users@postfix.org Subject: Re: warning: psc_dnsbl_request: connect to private/dnsblog service: Resource temporarily unavailable On Thu, Oct 23, 2014 at 05:53:01PM +0200, Laurens Van Acker wrote: # pgrep -x dnsblog | wc -l 0 Seems unlikely, below nestat reports dnsblog processes The lock file is there: # ls /var/spool/postfix/pid/ ... unix.dnsblog ... Which process has it locked? # netstat -anp --unix | grep dnsblog unix 2 [ ACC ] STREAM LISTENING 526425 10764/master private/dnsblog unix 2 [ ] DGRAM717894 17046/dnsblog unix 2 [ ] DGRAM716334 17041/dnsblog unix 2 [ ] DGRAM717846 17038/dnsblog unix 2 [ ] DGRAM716310 17037/dnsblog unix 2 [ ] DGRAM717882 17044/dnsblog unix 2 [ ] DGRAM716322 17039/dnsblog unix 2 [ ] DGRAM716346 17043/dnsblog unix 2 [ ] DGRAM717858 17040/dnsblog unix 2 [ ] DGRAM717870 17042/dnsblog unix 2 [ ] DGRAM716360 17045/dnsblog unix 2 [ ] DGRAM716520 17053/dnsblog She DGRAM socket is likely the syslog socket. I'm not seeing any dnsblog processes listening on the actual dnsblog socket. You have to dig deeper, but may lack the skill to do so... I can't spoon-feed you through this one step at a time. :-( -- Viktor.
RE: warning: psc_dnsbl_request: connect to private/dnsblog service: Resource temporarily unavailable
If have this in rsyslog.conf already: mail.* -/var/log/mail.log So I already disabled synchronous. Will try a reboot, thanks! Laurens -Original Message- From: owner-postfix-us...@postfix.org [mailto:owner-postfix-us...@postfix.org] On Behalf Of Viktor Dukhovni Sent: donderdag 23 oktober 2014 18:16 To: postfix-users@postfix.org Subject: Re: warning: psc_dnsbl_request: connect to private/dnsblog service: Resource temporarily unavailable On Thu, Oct 23, 2014 at 04:06:55PM +, Viktor Dukhovni wrote: She DGRAM socket is likely the syslog socket. I'm not seeing any dnsblog processes listening on the actual dnsblog socket. You have to dig deeper, but may lack the skill to do so... I can't spoon-feed you through this one step at a time. :-( Restarting Postfix or rebooting the machine may sweep the problem under the rug. Perhaps your syslog service is congested, if so, make sure you're not doing synchronous logging. -- Viktor.
Re: warning: psc_dnsbl_request: connect to private/dnsblog service: Resource temporarily unavailable
Viktor Dukhovni: On Thu, Oct 23, 2014 at 04:06:55PM +, Viktor Dukhovni wrote: She DGRAM socket is likely the syslog socket. I'm not seeing any dnsblog processes listening on the actual dnsblog socket. You have to dig deeper, but may lack the skill to do so... I can't spoon-feed you through this one step at a time. :-( Restarting Postfix or rebooting the machine may sweep the problem under the rug. Perhaps your syslog service is congested, if so, make sure you're not doing synchronous logging. Killing the dnsblog process that holds the lock may also jerk the kernel enoug. Wietse
Re: reject attachments for certain recipients
On 10/23/2014 10:44 AM, li...@rhsoft.net wrote: Am 23.10.2014 um 17:47 schrieb Andre Rodier: Is there a simple way in Postfix to reject attachments, but only for certain email addresses? no - not without policy daemons Selectively blocking attachments requires a content_filter, smtpd_proxy_filter, or milter. The policy service interface only sees envelope information and cannot detect attachments. -- Noel Jones
Re: Postfix/milter benchmarking
* Julian Mehnle jul...@mehnle.net: I'm developing a new milter and I need to benchmark it when plugged into Postfix. What are my best options for generating an SMTP stream of messages directed at a single Postfix instance at rates on the order of hundreds per second, and measuring SMTP acceptance rates and SMTP response latency? I can run processes on multiple machines if necessary, but the more efficient the better. mstone is a PITA to setup, but it works. swaks is a script, which you can create a wrapper around. Which fits better? Depends on the job your milter needs to do. What is its focus? If try to filter sessions, mstone will probably serve you better, since you can easily tune frequency, number of connections etc. If you need to simulate specific content swaks might be better. We've easily done 90 msg/sec with a milter that picks up session data and specific headers on standard server hardware. p@rick I saw the smtp-source tool http://www.postfix.org/smtp-source.1.html, but I'm not sure I can tune it to certain output rates and make it take acceptance rate and latency measurements. Are there any other tools people use to benchmark their Postfix setups or, more specifically, milters? Thanks, -Julian -- [*] sys4 AG https://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
Re: reject attachments for certain recipients
On October 23, 2014 5:47:39 PM Andre Rodier an...@rodier.me wrote: Is there a simple way in Postfix to reject attachments, but only for certain email addresses? http://sanesecurity.com/foxhole-databases/ Already better handled outside of postfix, so if you already use clamav this is done, if you want pr recipients the its possible with amavisd without clamav, but imho amavisd does not check in achives with clamav does This point is important when it comes to performance
Re: reject attachments for certain recipients
Am 23.10.2014 um 20:13 schrieb Benny Pedersen: On October 23, 2014 5:47:39 PM Andre Rodier an...@rodier.me wrote: Is there a simple way in Postfix to reject attachments, but only for certain email addresses? http://sanesecurity.com/foxhole-databases/ completly off-topic, don't get me wrong but you should sometimes read what you reply for (as well on dovecot / spamassassin list)
Re: Postfix/milter benchmarking
Julian Mehnle: Are there any other tools people use to benchmark their Postfix setups or, more specifically, milters? Wietse told on talk I listened Optimize both the worst case and the common case. Worst cases become normal cases When developing milter applicaions that mean to me: - decide as early as possible - allocate memory / do complex calculation as late as possible Andreas
Multikey/multivalue table lookups
Wietse Venema: Viktor Dukhovni: Sorry, it must be the same interface as access tables. Special cases are taboo. I'm not fond of lossful mangling of the input, especially if policy services are liable to get it wrong. Perhaps this idea needs to wait until the Postfix dictionary layer understands multiple results, returning a list rather than a concatenation. At that point, suitable variants of pcre/regexp or directives at the top of the file can turn on list splitting of the RHS prior to sub-pattern replacement. Postfix has a simple string interface for its key-value store. I have an aversion against changing that into some binary protocol with sub-record structure. Similar to my aversion against systemd logging, among other things. JSON can be represented as printable ASCII, and circumvent some of my aversion against binary protocols. But wait, there is more. Fundamentally, there is a high cost when switching from the current no encoding table lookup interfaces to some encoding: all Postfix tables would have to switch from the current unencoded interface to some encoded interface, based on JSON or something else. This is not a matter of adding a special-case entry at the top of some regexp or pcre file. Multikey/multivalue representation pervades all map interfaces. Exceptions are not allowed. This is what a single-key, multi-value, JSON-based system aliases database would look like: current: root: foo, bar becomes: root [foo,bar] To implement two-key lookup with Berkeley DB, we'd specify something like [xxx,yyy] as the real Berkeley DB database lookup key, where xxx is the first key and yyy the second one. In other words, it is faked with concatenation. The same encoding would be used internally in the Postfix SMTPD policy protocol queries and responses: the query as a list of (key, value) pairs, and the result as a list of values. Every policy server would have to switch to the new encoding. Now, suppose we have a single-key, multi-value access map entry in some PCRE table: /^(.+@example\.com)$/ [warn redirecting $1, redirect f...@example.net] (Presumably, two-key matching would require two PCRE patterns.) Here, the PCRE map would replace quotes and backslashes in the expansion of $number with JSON escape sequences, so that the expanded text cannot change the syntactical structure of the result. Wietse
Define exception(s) from catchall domain
Hello, I have a few users that insist on using catch-all domains. Not surprising they get spam to some address. Now they're asking if they can reject mail for *some* of the addresses of the catch-all domain. They can create aliases themselves via postfixadmin and they want to do this the same way. I tried to implement this by using a check_recipient_access pcre_table like this: /etc/postfix# cat recipient_access.pcre /^postfix-reject-address@.+$/ REJECT smtpd_recipient_restrictions = check_recipient_access pcre:$config_directory/recipient_access.pcre, ... And telling them to add an alias to postfix-reject-address@$THEIR_DOMAIN But this doesn't work as postfix will produce bounces (backscatter) like this: reject-postfix-addr...@karotte.org (expanded from reject-t...@karotte.org): user unknown In the log I see that postfix tries to deliver the message with the default virtual transport (dovecot) which then returns the user unknown. It there a way to acomplish this? Regards Sebastian -- GPG Key: 0x93A0B9CE (F4F6 B1A3 866B 26E9 450A 9D82 58A2 D94A 93A0 B9CE) 'Are you Death?' ... IT'S THE SCYTHE, ISN'T IT? PEOPLE ALWAYS NOTICE THE SCYTHE. -- Terry Pratchett, The Fifth Elephant
Re: Define exception(s) from catchall domain
Am 23.10.2014 um 21:52 schrieb Sebastian Wiesinger: I have a few users that insist on using catch-all domains. Not surprising they get spam to some address. Now they're asking if they can reject mail for *some* of the addresses of the catch-all domain. They can create aliases themselves via postfixadmin and they want to do this the same way. I tried to implement this by using a check_recipient_access pcre_table like this: /etc/postfix# cat recipient_access.pcre /^postfix-reject-address@.+$/ REJECT smtpd_recipient_restrictions = check_recipient_access pcre:$config_directory/recipient_access.pcre, ... And telling them to add an alias to postfix-reject-address@$THEIR_DOMAIN But this doesn't work as postfix will produce bounces (backscatter) like this: reject-postfix-addr...@karotte.org (expanded from reject-t...@karotte.org): user unknown In the log I see that postfix tries to deliver the message with the default virtual transport (dovecot) which then returns the user unknown. It there a way to acomplish this? smtpd_recipient_restrictions with REJECT do NOT backscatter a proper REJECT in the MTA never send a bounce if it touchs the virtual transport the REJECT never got triggered i do not see postconf -n output nor a full log example for such a message, so it's impossible to know what happens on your setup anyways, somebody insisting in a catch-all in 2014 has to suck the spam or give up that completly broken idea - it even did not made sense 15 years ago - if somebody don't know my address he can#t send a mail to me - so what - would you extend that to @internet - no - so why to @domain?
Re: Define exception(s) from catchall domain
* Sebastian Wiesinger postfix-us...@ml.karotte.org [2014-10-23 21:54]:
Hello,
I have a few users that insist on using catch-all domains. Not
surprising they get spam to some address. Now they're asking if they
can reject mail for *some* of the addresses of the catch-all domain.
They can create aliases themselves via postfixadmin and they want to
do this the same way.
I tried to implement this by using a check_recipient_access pcre_table
like this:
/etc/postfix# cat recipient_access.pcre
/^postfix-reject-address@.+$/ REJECT
smtpd_recipient_restrictions =
check_recipient_access pcre:$config_directory/recipient_access.pcre,
...
And telling them to add an alias to
postfix-reject-address@$THEIR_DOMAIN
But this doesn't work as postfix will produce bounces (backscatter)
like this:
reject-postfix-addr...@karotte.org (expanded from
reject-t...@karotte.org):
user unknown
Forgot the logs/configuration:
postconf -n:
alias_database = hash:/etc/aliases
alias_maps = hash:/etc/aliases
append_dot_mydomain = no
biff = no
body_checks = pcre:$config_directory/body_checks.pcre
broken_sasl_auth_clients = yes
config_directory = /etc/postfix
disable_vrfy_command = yes
dovecot-sa_destination_recipient_limit = 1
dovecot_destination_recipient_limit = 1
enable_long_queue_ids = yes
greylist = check_policy_service inet:127.0.0.1:10023
home_mailbox = Maildir/
html_directory = /usr/share/doc/postfix/html
inet_interfaces = 127.0.0.1, [::1], 176.9.75.247, 176.9.51.79,
[2a01:4f8:150:7142::25], [2a01:4f8:150:7142::587]
inet_protocols = ipv4, ipv6
mailbox_command = /usr/bin/procmail -a $EXTENSION
mailbox_size_limit = 0
message_size_limit = 10240
mydestination = mx.karotte.org, alita.karotte.org, localhost.karotte.org,
localhost
myhostname = mx.karotte.org
mynetworks = 127.0.0.0/8 [:::127.0.0.0]/104 [::1]/128
myorigin = /etc/mailname
non_smtpd_milters = inet:127.0.0.1:10100, inet:127.0.0.1:10101
parent_domain_matches_subdomains =
recipient_delimiter = +
relay_clientcerts = hash:$config_directory/relay_clientcerts
relay_domains = proxy:mysql:$config_directory/sql/mysql_relay_domains_maps.cf
relayhost =
smtp_address_preference = ipv6
smtp_bind_address = 176.9.75.247
smtp_bind_address6 = 2a01:4f8:150:7142::25
smtp_dns_support_level = dnssec
smtp_tls_CApath = /etc/ssl/certs
smtp_tls_fingerprint_digest = sha1
smtp_tls_loglevel = 1
smtp_tls_mandatory_protocols = !SSLv2 !SSLv3
smtp_tls_policy_maps = hash:$config_directory/tls_policy
smtp_tls_security_level = dane
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtpd_client_connection_count_limit = 5
smtpd_client_connection_rate_limit = 15
smtpd_client_event_limit_exceptions = $mynetworks, $inet_interfaces
smtpd_client_restrictions = permit_mynetworks, permit_inet_interfaces,
permit_sasl_authenticated, permit_tls_clientcerts, check_client_access
cidr:$config_directory/unknown_reverse_hostname.cidr, check_client_access
hash:$config_directory/client_rbl_whitelist, permit_dnswl_client
list.dnswl.org=127.0.[0..255].[1..3], reject_rbl_client
zen.spamhaus.org=127.0.0.[2..11], reject_rbl_client ix.dnsbl.manitu.net,
reject_rhsbl_reverse_client dbl.spamhaus.org=127.0.1.[2;4..6]
smtpd_data_restrictions = reject_unauth_pipelining
smtpd_delay_reject = yes
smtpd_discard_ehlo_keywords = silent-discard, dsn
smtpd_etrn_restrictions = reject
smtpd_helo_required = yes
smtpd_helo_restrictions = permit_mynetworks, permit_inet_interfaces,
permit_sasl_authenticated, permit_tls_clientcerts,
reject_invalid_helo_hostname, reject_non_fqdn_helo_hostname,
reject_rhsbl_helo dbl.spamhaus.org=127.0.1.[2;4..6]
smtpd_milters = inet:127.0.0.1:10100, inet:127.0.0.1:10101
smtpd_recipient_restrictions = check_recipient_access
pcre:$config_directory/recipient_access.pcre, permit_mynetworks,
permit_inet_interfaces, reject_non_fqdn_recipient,
permit_sasl_authenticated, permit_tls_clientcerts, check_recipient_access
hash:$config_directory/defer_unkown_users, reject_unlisted_recipient,
check_policy_service unix:private/policyd-spf, permit_dnswl_client
list.dnswl.org=127.0.[0..255].[0..3], check_recipient_access
pcre:$config_directory/greylist.pcre
smtpd_relay_restrictions = permit_mynetworks, permit_inet_interfaces,
permit_sasl_authenticated, permit_tls_clientcerts, reject_unauth_destination
smtpd_restriction_classes = greylist
smtpd_sasl_auth_enable = yes
smtpd_sasl_path = private/auth
smtpd_sasl_security_options = noanonymous
smtpd_sasl_tls_security_options = noanonymous
smtpd_sasl_type = dovecot
smtpd_sender_restrictions = permit_mynetworks, permit_inet_interfaces,
reject_non_fqdn_sender, permit_sasl_authenticated, permit_tls_clientcerts,
reject_unlisted_sender, reject_unknown_sender_domain, reject_rhsbl_sender
dbl.spamhaus.org=127.0.1.[2;4..6]
smtpd_tls_auth_only = yes
smtpd_tls_cert_file = /etc/ssl/certs/cacert-karotte-combined.crt
smtpd_tls_dh1024_param_file =
Re: Define exception(s) from catchall domain
On 10/23/2014 2:52 PM, Sebastian Wiesinger wrote: Hello, I have a few users that insist on using catch-all domains. Not surprising they get spam to some address. Now they're asking if they can reject mail for *some* of the addresses of the catch-all domain. They can create aliases themselves via postfixadmin and they want to do this the same way. I tried to implement this by using a check_recipient_access pcre_table like this: /etc/postfix# cat recipient_access.pcre /^postfix-reject-address@.+$/ REJECT This must match the recipient address as sent by the client and logged by postfix smtpd process, NOT the rewritten address. smtpd_recipient_restrictions = check_recipient_access pcre:$config_directory/recipient_access.pcre, ... It's generally unwise to put any access tables before permit_mynetworks. Extra caution is needed to make sure you don't accidentally create an open relay. http://www.postfix.org/SMTPD_ACCESS_README.html#danger And telling them to add an alias to postfix-reject-address@$THEIR_DOMAIN This should not be necessary. -- Noel Jones
Re: Multikey/multivalue table lookups
On Thu, Oct 23, 2014 at 03:02:08PM -0400, Wietse Venema wrote:
Postfix has a simple string interface for its key-value store. I
have an aversion against changing that into some binary protocol
with sub-record structure. Similar to my aversion against systemd
logging, among other things.
JSON can be represented as printable ASCII, and circumvent some of
my aversion against binary protocols. But wait, there is more.
Being somewhat old-fashioned, my initial mental model of multi-value
lists was TCL lists, somewhat simpler than JSON. But that's just
a user-interface question. Tcl lists are simpler when the elements
are just words, but JSON has a lot more tooling and mind-share.
The question is then whether to optimize for ease of machine-parsing
or ease of human editing, and which is the better syntax for one
or the other.
Simple 3-element Tcl list:
abc def 123
Tcl list of two lists:
{abc def 123} {xyzzy elbereth}
current:
root: foo, bar
becomes:
root [foo,bar]
Tcl (key, value) list:
root{foo bar}
To implement two-key lookup with Berkeley DB, we'd specify something
like [xxx,yyy] as the real Berkeley DB database lookup key,
where xxx is the first key and yyy the second one. In other
words, it is faked with concatenation.
Tcl list with a two-element key and a three-element value:
{key1 key2} {val1 val2 val3}
Now, suppose we have a single-key, multi-value access map entry
in some PCRE table:
/^(.+@example\.com)$/ [warn redirecting $1, redirect f...@example.net]
Here, the PCRE map would replace quotes and backslashes in the
expansion of $number with JSON escape sequences, so that the expanded
text cannot change the syntactical structure of the result.
Yes, the dictionary layer would have to honour the list structure.
This is big-time incompatible, since existing RHS values with
white-space would have to be enclosed in some sort of syntax to
indicate a single entity containing white-space.
--
Viktor.
Re: Multikey/multivalue table lookups
Viktor Dukhovni: On Thu, Oct 23, 2014 at 03:02:08PM -0400, Wietse Venema wrote: Postfix has a simple string interface for its key-value store. I have an aversion against changing that into some binary protocol with sub-record structure. Similar to my aversion against systemd logging, among other things. JSON can be represented as printable ASCII, and circumvent some of my aversion against binary protocols. But wait, there is more. Being somewhat old-fashioned, my initial mental model of multi-value lists was TCL lists, somewhat simpler than JSON. But that's just a user-interface question. Indeed. The issues that I raised are that 1) support for multiple values or multiple results will require some kind of encoding, and 2) this encoding must be robust enough that regexp/pcre tables can safely substitute text into them without changing the over-all syntactical structure. A third issue is that other programs must be able to process Postfix tables and implement Postfix policy servers with readily-available parsers. The current free-text format is so brain-dead simple that no custom parser code is needed. As for structured formats, JSON is a standard feature of many languages, TCL is not. Wietse
