Re: Relay email based on Sender address / domain
Hi!! > sender_dependent_default_transport_maps = > hash:$config_directory/sender_dependant_relayhost_maps.db Here you should point to the plain text file, not the db. The db file should be generated by running the postmap command: # postmap $config_directory/sender_dependant_relayhost_maps > however the instance continues to deliver email via the internet route Vs > route via a GW . > > i have not disabled transport_maps as we are using it to give direction to > incoming NDRs in a multi instance setup. During the next hop lookup the transport_maps has priority over everything, so if you have a wildcard there for example, your sender based configuration will never be reached. Cheers!
Re: PATCH: Postfix benchmark: bug or performance regression ?
On Friday, 29 March 2019 00:49:34 CET Wietse Venema wrote: > > This is a patch for LMTP connection reuse over UNIX-domain sockets. > > Bugfix (introduced: Postfix 3.0): LMTP connections over > UNIX-domain sockets were cached but not reused, due to a > cache lookup key mismatch. Therefore, idle cached connections > could exhaust LMTP server resources, resulting in two-second > pauses between email deliveries. This problem was investigated > by Juliana Rodrigueiro. File: smtp/smtp_connect.c. > > These diffs are identical except that in Postfix 3.4 some macros > were renamed to better match their purpose. > > Tested with Postfix 3.5. Please give it a try. > > wietse There is a small problem in the 3.1-3.3.diff though. I believe the variable "dest" should actually be "path". I quickly tested with this modifications in postfix 3.3.2 and everything worked. Thank you a lot for the patch! Julie.
Re: Postfix benchmark: bug or performance regression ?
On Wednesday, 27 March 2019 20:01:49 CET Viktor Dukhovni wrote: > On Wed, Mar 27, 2019 at 03:36:28PM +0100, Juliana Rodrigueiro wrote: > > However, during a benchmark, we realized 3.3.2 was 5 times slower than the > > version before. > > This is misleading. Postfix is not 5 times slower, your benchmark > appears to be measuring the LMTP delivery rate to a single sink > mailbox, and this artificial work-load may be slower under some > adverse conditions (below...). > Although the benchmark produces a very large artificial work-load, this scenario is still a very valid use case, even more considering that 20 emails are enough to clog up deliveries. The stark difference between the LMTP server (cyrus 2.4) log messages is that most deliveries reuse the same session id when running postfix <= 2.11.0: Mar 28 10:22:22 localdomain lmtpunix[4881]: auditlog: append sessionid= mailbox= uniqueid=<3aab65ba5c98d020> uid=<28047> guid=<521e5e4a35095385881941a77f3035f6087c8942> Mar 28 10:22:22 localdomain lmtpunix[4881]: Delivered: <20190328092221.f2f6...@localdomain.com> to mailbox: user.dst Mar 28 10:22:22 localdomain lmtpunix[4881]: USAGE dst user: 0.00 sys: 0.00 Mar 28 10:22:22 localdomain lmtpunix[4926]: auditlog: append sessionid= mailbox= uniqueid=<3aab65ba5c98d020> uid=<28048> guid= Mar 28 10:22:22 localdomain lmtpunix[4926]: Delivered: <2019032809.0044...@localdomain.com> to mailbox: user.dst ... ... Mar 28 10:22:22 localdomain lmtpunix[4926]: auditlog: append sessionid= mailbox= uniqueid=<3aab65ba5c98d020> uid=<28060> guid=<0dc9e922f8b8cfc01ccc5bfeafa8220812adf8fa> Mar 28 10:22:22 localdomain lmtpunix[4926]: Delivered: <2019032809.0da1...@localdomain.com> to mailbox: user.dst Mar 28 10:22:22 localdomain lmtpunix[4926]: USAGE dst user: 0.00 sys: 0.004000 Mar 28 10:22:22 localdomain lmtpunix[6449]: auditlog: append sessionid= mailbox= uniqueid=<3aab65ba5c98d020> uid=<28061> guid=<781d02ef98dde39e03461a60aa18beb858f8a5d2> Mar 28 10:22:22 localdomain lmtpunix[6449]: Delivered: <2019032809.0ea9...@localdomain.com> to mailbox: user.dst In the other hand, for version >= 2.11.1, the sessions are used once per process. Mar 28 10:04:06 localdomain lmtpunix[4273]: Delivered: <20190328090404.7cf8...@localdomain.com> to mailbox: user.dst Mar 28 10:04:06 localdomain lmtpunix[4273]: USAGE dst user: 0.004000 sys: 0.00 Mar 28 10:04:06 localdomain lmtpunix[4292]: auditlog: append sessionid= mailbox= uniqueid=<3aab65ba5c98d020> uid=<28032> guid=<165f88d841750f2cc9a59a74e777ada4c884dae3> Mar 28 10:04:06 localdomain lmtpunix[4292]: Delivered: <20190328090404.7dee...@localdomain.com> to mailbox: user.dst Mar 28 10:04:06 localdomain lmtpunix[4292]: USAGE dst user: 0.00 sys: 0.00 Mar 28 10:04:06 localdomain lmtpunix[4295]: auditlog: append sessionid= mailbox= uniqueid=<3aab65ba5c98d020> uid=<28033> guid= ... ... Mar 28 10:04:10 localdomain lmtpunix[4295]: auditlog: append sessionid= mailbox= uniqueid=<3aab65ba5c98d020> uid=<28041> guid=<7ba5808ec13795785ab5a5737cfef97ff24d307a> Mar 28 10:04:10 localdomain lmtpunix[4295]: Delivered: <20190328090404.87ac...@localdomain.com> to mailbox: user.dst Mar 28 10:04:10 localdomain lmtpunix[4295]: USAGE dst user: 0.00 sys: 0.004000 Mar 28 10:04:12 localdomain lmtpunix[4273]: auditlog: append sessionid= mailbox= uniqueid=<3aab65ba5c98d020> uid=<28042> guid=<8ee41c08e69e9cdf9be61b2f2bad1a2d2c590f12> Mar 28 10:04:12 localdomain lmtpunix[4273]: Delivered: <20190328090404.88ae...@localdomain.com> to mailbox: user.dst > > Mar 27 14:46:58 localdomain postfix/lmtp[24750]: 7CFDE72: > > to=, orig_to=, > > relay=localdomain.com[/var/ imap/socket/lmtp], delay=8.1, > > delays=0/6/2/0.02, dsn=2.1.5, status=sent (250 > This delivery spent 6s waiting in the active queue and 2s to complete > a connection. Why are connections to the LMTP server so expensive? > Is your syslog service logging synchronously? Perhaps logging of > connection establishment (in the LMTP server) is causing congestion > at LMTP connect? What is the LMTP server doing in connect and LHLO > processing? > Unlikely, the LMTP server doesn't log any more or less than it did before. > > -#define HAVE_NEXTHOP_STATE(state) > > (STR((state)->iterator->request_nexthop) != 0) > > +#define HAVE_NEXTHOP_STATE(state) > > (STR((state)->iterator->request_nexthop)[0] != 0) > That's a necessary bugfix, without it the test is always true. > I understand the necessary fix, and that is why I was puzzled by the behaviour change. > > Questions: > > > > (1) Does it mean that the lmtp connections via socket are never cached or > > reused? >
Postfix benchmark: bug or performance regression ?
Hi all! We used to have postfix 2.6.11 in our systems, which was then updated with no problems to 3.3.2. However, during a benchmark, we realized 3.3.2 was 5 times slower than the version before. Even after disabling all mail filters the slowdown was still the same. The benchmark consists of a large amount of emails being fetched and entering postfix via the smtpd daemon and then handed over to cyrus via lmtp and socket to be delivered mostly to the same recipient. At first, we thought maybe the concurrency limits have changed, or maybe now there are less processes running, but all the configuration variables regarding this have not changed. The benchmark uses 3 emails, but generating 20 emails is sufficient to see the bug. After trials and errors and running out of ideas, I decided to bisect the code to see what made this change. It happens that version 2.11.1 behaves as slow as 3.3.2, and version 2.11.0 as fast as 2.6.11. All with the same hardware, same configuration, same load. With version >= 2.11.1, the first emails are received and processed somewhat instantly, after some time it seems that lmtp is not processing anything and just waiting while all emails are fetched, so afterwards it starts delivering to the recipients at the rate of about 2 per second. (See excerpt of the log file) Excerpt of maillog version > 2.11.1: Mar 27 14:46:50 localdomain postfix/qmgr[24339]: 7079468: from=, size=356, nrcpt=1 (queue active) Mar 27 14:46:50 localdomain postfix/lmtp[24750]: 6CEFF61: to=, orig_to=, relay=localdomain.com[/var/ imap/socket/lmtp], delay=0.02, delays=0.01/0/0.01/0, dsn=2.1.5, status=sent (250 2.1.5 Ok SESSIONID=) Mar 27 14:46:50 localdomain postfix/smtpd[24746]: 718F46A: client=localdomain.com[127.0.0.1] Mar 27 14:46:50 localdomain postfix/qmgr[24339]: 6CEFF61: removed Mar 27 14:46:50 localdomain postfix/cleanup[24748]: 718F46A: message- id=<20190327134650.718f...@localdomain.com> Mar 27 14:46:50 localdomain postfix/qmgr[24339]: 718F46A: from=, size=356, nrcpt=1 (queue active) Mar 27 14:46:50 localdomain postfix/smtpd[24746]: 7273461: client=localdomain.com[127.0.0.1] Mar 27 14:46:50 localdomain postfix/lmtp[24752]: 6E8CE63: to=, orig_to=, relay=localdomain.com[/var/ imap/socket/lmtp], delay=0.02, delays=0/0/0.01/0.01, dsn=2.1.5, status=sent (250 2.1.5 Ok SESSIONID=) Mar 27 14:46:50 localdomain postfix/qmgr[24339]: 6E8CE63: removed ... ... Mar 27 14:46:50 localdomain postfix/cleanup[24748]: 764A06A: message- id=<20190327134650.764a...@localdomain.com> Mar 27 14:46:50 localdomain postfix/qmgr[24339]: 764A06A: from=, size=356, nrcpt=1 (queue active) Mar 27 14:46:50 localdomain postfix/smtpd[24746]: 778DC61: client=localdomain.com[127.0.0.1] Mar 27 14:46:50 localdomain postfix/cleanup[24758]: 778DC61: message- id=<20190327134650.778d...@localdomain.com> Mar 27 14:46:50 localdomain postfix/qmgr[24339]: 778DC61: from=, size=356, nrcpt=1 (queue active) Mar 27 14:46:50 localdomain postfix/smtpd[24746]: 786A96B: client=localdomain.com[127.0.0.1] Mar 27 14:46:50 localdomain postfix/cleanup[24758]: 786A96B: message- id=<20190327134650.786a...@localdomain.com> ... ... Mar 27 14:46:52 localdomain postfix/lmtp[24752]: 764A06A: to=, orig_to=, relay=localdomain.com[/var/ imap/socket/lmtp], delay=2, delays=0/2/0/0, dsn=2.1.5, status=sent (250 2.1.5 Ok SESSIONID=) Mar 27 14:46:52 localdomain postfix/qmgr[24339]: 764A06A: removed Mar 27 14:46:54 localdomain postfix/lmtp[24750]: 778DC61: to=, orig_to=, relay=localdomain.com[/var/ imap/socket/lmtp], delay=4, delays=0.01/2/2/0.02, dsn=2.1.5, status=sent (250 2.1.5 Ok SESSIONID=) Mar 27 14:46:54 localdomain postfix/qmgr[24339]: 778DC61: removed ... ... Mar 27 14:46:58 localdomain postfix/lmtp[24750]: 7CFDE72: to=, orig_to=, relay=localdomain.com[/var/ imap/socket/lmtp], delay=8.1, delays=0/6/2/0.02, dsn=2.1.5, status=sent (250 2.1.5 Ok SESSIONID=) Mar 27 14:46:58 localdomain postfix/qmgr[24339]: 7CFDE72: removed With version <= 2.11.0, the messages are delivered as soon as they arrive. Excerpt of maillog version <= 2.11.0: ... ... Mar 27 15:00:35 localdomain postfix/qmgr[26443]: AA6CC67: removed Mar 27 15:00:35 localdomain postfix/smtpd[26446]: AC80663: client=localdomain.com[127.0.0.1] Mar 27 15:00:35 localdomain postfix/cleanup[26456]: AC80663: message- id=<20190327140035.ac80...@localdomain.com> Mar 27 15:00:35 localdomain postfix/qmgr[26443]: AC80663: from=, size=356, nrcpt=1 (queue active) Mar 27 15:00:35 localdomain postfix/lmtp[26451]: AB58261: to=, orig_to=, relay=localdomain.com[/var/ imap/socket/lmtp], conn_use=2, delay=0.01, delays=0/0/0/0, dsn=2.1.5, status=sent (250 2.1.5 Ok SESSIONID=) Mar 27 15:00:35 localdomain postfix/smtpd[26446]: AD4EB67: client=localdomain.com[127.0.0.1] Mar 27 15:00:35 localdomain postfix/qmgr[26443]: AB58261: removed Mar 27 15:00:35 localdomain postfix/cleanup[26456]: AD4EB67: message- id=<20190327140035.ad4e...@localdomain.com> Mar 27 15:00:35 localdomain
