Re: SMTPS 465
Am 2013-04-15 07:27 schrieb Stan Hoeppner: We've been told that this kernel upgrade created the problem. Simply reverting to the previous kernel should fix it. I wager the OP upgraded more than just the kernel, or upgraded the kernel in-place, Russian Roulette style. ;) The lesson learned is that package managers are a Good Thing, and if you really need to roll out custom-built software, it pays to create your own packages, especially if you can build them with a chrooted builder in a clean environment.
Re: SMTPS 465
On 2013-04-14 6:30 PM, Joan Moreau j...@grosjo.net wrote: Le 14/04/2013 22:24, Viktor Dukhovni a écrit : On Sun, Apr 14, 2013 at 10:21:58PM +, Joan Moreau wrote: However, how can postfix NOT use the only openssl library ? or fail to have SHA2 when loading the .so ? Find a less broken operating system. This works on every system I've ever used, and finding out what's wrong with yours is not a good use of your time or mine. Well, this server has worked since ever, supporting plenty of web operations (so I can not really 'delete and re-install' and broke only after updating the kernel. Any other clue ? Roll back to the previous kernel. Seriously. If you updated the kernel but didn't keep the last known good/working one, then hopefully you have learned why doing this is such a good idea and will do so in the future. -- Best regards, Charles
Re: postfix and Berkeley DB
On 13 Apr 2013, at 09:29 , Sahil Tandon sahil+post...@tandon.net wrote: None. Postfix is using libc, which appears in your ldd(1) output, and contains the Berkeley DB 1.85 routines. Ah-hah, thank you for that. --
R: Scheduling policies for outgoing smtp server
Hi, I'm sorry for the delay, but I'm very busy in some projects. I continue the discussion with my opinion and some details. No virtual machines and no multi instance solution: we have more than 10.000 customers, so these solutions are not applicable. We don't want to classify them (in order to have fewer outgoing queues). I'm thinking and trying to found a solution that could respect the indication Postfix mustn't fail with more than N messages in the queue. I work for an ISP, so I consider incoming emails and outgoing emails as two separate services. The first is already served in the right way by the existing scheduling algorithm, for the second I think that there could be another separated scheduling algorithm. So I think about the possibility to choose a different algorithm by some configuration parameter (i.e. specifying in master.cf startup different options for different transports). How does Postfix enforce per-destination concurrency policy, when mail is kept in per-customer queues? Assuming that we choose the right algorithm for every transport (the default could be the existing one): - with incoming algorithm (the existing one): there is no need for sender round-robin, in fact the current algorithm doesn't use sender information - with outgoing algorithm: it should be analogous; in this case we need to do only sender round-robin, the destination doesn't matter. The outgoing scheduling algorithm looks like the incoming with the difference in the step of picking up the email (like I wrote in a previous post). How can we implement round-robin by sender ip/authenticated user and to preserve the memory constraint too? - sender is the sender's ip address or the authenticated user name (i.e. 80.93.143.50 or giorgio.luchi) - rrsender_message_limit is the max number of messages in the sender queue (i.e. 3) - rrsender_queue_limit is the max number of senders actually loaded in RAM (i.e. 6000) - What happens when Postfix opens a queue file and discovers that this customer's queue is full? Postfix must not open the same files again and again in an infinite loop, and Postfix must not have foreach job in the queue knowledge. - We could move such messages to the deferred queue, but that would make queue management more expensive, replacing rename(incoming, active) +remove() with rename(incoming, deferred) +rename(deferred, active) +remove(): I think the solution should behaves as the second proposal. I know that it costs more in term of IOPS, but in our environment of ISP, we can spend some effort and money for this operation and accepting lower performance (we can gain the lost power, balancing the load with several dedicated servers) - so, at any time, in RAM there will be at maximum rrsender_queue_limit queues per rrsender_message_limit messages, it means 6000 different senders that need to send emails simultaneously (I think is a big number) - we could also implement preemption in an analogous way as the current scheduling manager do (if a message is sent to several recipients) I hope this can help in understanding and in finding a solution to what we need. Regards Giorgio Luchi
Re: SMTPS 465
Le 15/04/2013 10:24, Charles Marcus a écrit : On 2013-04-14 6:30 PM, Joan Moreau j...@grosjo.net wrote: Le 14/04/2013 22:24, Viktor Dukhovni a écrit : On Sun, Apr 14, 2013 at 10:21:58PM +, Joan Moreau wrote: However, how can postfix NOT use the only openssl library ? or fail to have SHA2 when loading the .so ? Find a less broken operating system. This works on every system I've ever used, and finding out what's wrong with yours is not a good use of your time or mine. Well, this server has worked since ever, supporting plenty of web operations (so I can not really 'delete and re-install' and broke only after updating the kernel. Any other clue ? Roll back to the previous kernel. Seriously. If you updated the kernel but didn't keep the last known good/working one, then hopefully you have learned why doing this is such a good idea and will do so in the future. Reverted to 3.7.10. Recompiled openssl + cyrus + posfix . Same errors. Where does the inconsistency reside ? 2013-04-15T13:55:29.921960+02:00 server postfix/smtpd[3308]: warning: TLS library problem: 3308:error:1411C146:SSL routines:tls1_prf:unsupported digest type:t1_enc.c:276: 2013-04-15T13:55:29.921966+02:00 server postfix/smtpd[3308]: warning: TLS library problem: 3308:error:140D308A:SSL routines:TLS1_SETUP_KEY_BLOCK:cipher or hash unavailable:t1_enc.c:597:
Re: SMTPS 465
Am 15.04.2013 13:57, schrieb Joan Moreau: Le 15/04/2013 10:24, Charles Marcus a écrit : Roll back to the previous kernel. Seriously. If you updated the kernel but didn't keep the last known good/working one, then hopefully you have learned why doing this is such a good idea and will do so in the future. Reverted to 3.7.10. Recompiled openssl + cyrus + posfix . Same errors. Where does the inconsistency reside ? 2013-04-15T13:55:29.921960+02:00 server postfix/smtpd[3308]: warning: TLS library problem: 3308:error:1411C146:SSL routines:tls1_prf:unsupported digest type:t1_enc.c:276: 2013-04-15T13:55:29.921966+02:00 server postfix/smtpd[3308]: warning: TLS library problem: 3308:error:140D308A:SSL routines:TLS1_SETUP_KEY_BLOCK:cipher or hash unavailable:t1_enc.c:597 for me this sound like you are doing configure make make install well this does not cleanup all garbage, doing so repeatly let's as example fail mysql to compile at all this is why things like rpmbuild was invited, they are working in a predictable and clean buildroot and with the resulting binaries you can predictable downgrade/upgrade packages and since dependencies for libraries and APIs are automatically injected in the RPM packages you normally recognize a binary incompatible update because RPM refuse to install the package additionally NOBODY builds his binaries on the production machine and fires make install, NOBODY - this has to be done on a testmachine and if the binary package works there you minimize the risk i know that this does not help you very much now but that should be what you learned the hardway by brick a production environement without take care how to make sure updates are working however, this is not a postfix-problem, this is revenge of bad practice signature.asc Description: OpenPGP digital signature
File descriptor issue in Solaris 11.1 ?
Since we have upgraded to Solaris 11.1 from Solaris 11viathe IPS repository, we are observing the following in our Postfix smtp mail relay logs: Apr 15 13:27:34 smtp-server postfix/smtp[25213]: [ID 947731 mail.crit] fatal: stream_connect: send file descriptor: Resource temporarily unavailable Apr 15 13:27:40 smtp-server postfix/error[27918]: [ID 947731 mail.crit] fatal: stream_connect: send file descriptor: Resource temporarily unavailable Apr 15 13:27:40 smtp-server postfix/qmgr[27904]: [ID 947731 mail.crit] fatal: stream_connect: send file descriptor: Resource temporarily unavailable Apr 15 13:27:40 smtp-server postfix/error[27585]: [ID 947731 mail.crit] fatal: stream_connect: send file descriptor: Resource temporarily unavailable Apr 15 13:27:40 smtp-server postfix/error[27855]: [ID 947731 mail.crit] fatal: stream_connect: send file descriptor: Resource temporarily unavailable Apr 15 13:27:46 smtp-server postfix/cleanup[26564]: [ID 947731 mail.crit] fatal: stream_connect: send file descriptor: Resource temporarily unavailable Apr 15 13:27:46 smtp-server postfix/smtp[25129]: [ID 947731 mail.crit] fatal: stream_connect: send file descriptor: Resource temporarily unavailable Also getting the following error mails from Mailer-Daemon: === From: MAILER-DAEMON@smtp-relay(Mail Delivery System) To: postmaster@smtp-relay (Postmaster) Subject: Postfix SMTP server: errors from unknown Transcript of session follows. Out: 220 smtp-relay ESMTP Postfix In: EHLO MarlizeTHINK Out: 250-smtp-relay Out: 250-PIPELINING Out: 250-SIZE 10240 Out: 250-ETRN Out: 250-ENHANCEDSTATUSCODES Out: 250-8BITMIME Out: 250 DSN In: MAIL FROM: fxxxans@munged.domain Out: 250 2.1.0 Ok In: RCPT TO: hf.n...@foo.bar NOTIFY=SUCCESS,FAILURE,DELAY Out: 250 2.1.5 Ok In: DATA Out: 354 End data with CRLF.CRLF Out: 451 4.3.0 Error: queue file write error In: QUIT Out: 221 2.0.0 Bye For other details, see the local mail logfile === According to searches in Google, this points to a file descriptor issue. But as this is pre-Solaris 8 and make use of /dev/poll, this should not be the issue here? Also this was working fine on Solaris 10 and the first release of Solaris 11. The resource limits of the postfix master process and child processes indicating the number of file descriptors: smtp-server:~# plimit `pgrep master` 13019:/usr/libexec/postfix/master resource current maximum time(seconds)unlimitedunlimited file(blocks)unlimitedunlimited data(kbytes)unlimitedunlimited stack(kbytes)8192unlimited coredump(blocks)unlimitedunlimited nofiles(descriptors)6553665536 vmemory(kbytes)unlimitedunlimited smtp-server:~# plimit 29714 29714:cleanup -z -t unix -u resource current maximum time(seconds)unlimitedunlimited file(blocks)2020 data(kbytes)unlimitedunlimited stack(kbytes)8192unlimited coredump(blocks)unlimitedunlimited nofiles(descriptors)6553665536 vmemory(kbytes)unlimitedunlimited Any insight/advice will be appreciated here. Regards -- --- Jaco Lesch SAIX HLS Email: ja...@saix.net
Re: SMTPS 465
On Apr 15, 2013, at 13:57, Joan Moreau j...@grosjo.net wrote: Le 15/04/2013 10:24, Charles Marcus a écrit : On 2013-04-14 6:30 PM, Joan Moreau j...@grosjo.net wrote: Le 14/04/2013 22:24, Viktor Dukhovni a écrit : On Sun, Apr 14, 2013 at 10:21:58PM +, Joan Moreau wrote: However, how can postfix NOT use the only openssl library ? or fail to have SHA2 when loading the .so ? Find a less broken operating system. This works on every system I've ever used, and finding out what's wrong with yours is not a good use of your time or mine. Well, this server has worked since ever, supporting plenty of web operations (so I can not really 'delete and re-install' and broke only after updating the kernel. Any other clue ? Roll back to the previous kernel. Seriously. If you updated the kernel but didn't keep the last known good/working one, then hopefully you have learned why doing this is such a good idea and will do so in the future. Reverted to 3.7.10. Recompiled openssl + cyrus + posfix . Same errors. Where does the inconsistency reside ? Probably in library versions you updated along the way, changes in the way you compile things, and so on. Besides, aren't the odd kernel versions such as 3.5.x, 3.7.x etc. development kernels? I really don't get why you are experimenting with development level software on what you claim is a production system. I suggest you stop wasting everyone's time, including your own, and roll back to the supported production kernel for your distribution, supported library versions, and so on. If you need custom Postfix packages for some reason, compile packages against the supported libraries. And if you do want to experiment with development level software, do so on a system that you can wipe and reinstall, or a virtual machine that you can roll back to a previous snapshot. Also, own your decision to live on the cutting edge. Fixing it is YOUR problem when you do, don't outsource it to others. Cya, Jona
Re: SMTPS 465
Am 15.04.2013 14:14, schrieb DTNX Postmaster: Besides, aren't the odd kernel versions such as 3.5.x, 3.7.x etc. development kernels? why should they? since kernel 2.6 released around 10 years ago the versioning is no longer this way and 3.0.x is only a renumbering from 2.6.40 https://www.kernel.org/ stable: 3.8.7 stable: 3.7.10 [EOL] signature.asc Description: OpenPGP digital signature
Case sensivity: Strict rfc5321 or reality compliance
Hi, sorry, I know this is not directly related to postfix but I know that there are several very experienced people reading this list. My question is how you (the people that use and administer mailservers) handle the localpart case sensivity according to rfc5321: The local-part of a mailbox MUST BE treated as case sensitive. Background ist the development of postfwd rate limiting, which allows amongst others to place maximum values (count or volume) for sender or recipient addresses. To do so postfwd creates a hash with the given address as index. At the moment it is easy to trick such limits by varying the case of the addresses, meaning that postfwd will create different counters for bob@alice.local, BoB@alice.local and bob@AlicE.local. No question - with future versions postfwd will definitely treat the domainpart case-insensitive, so that bob@alice.local and bob@AlicE.local will have the same counter. But for the localpart I see two options, wondering which I should choose: 1.) Being strictly rfc5321 compliant: By default postfwd creates two different counters for bob@alice.local and BoB@alice.local. To change this the admin has to specify --non-rfc5321-rate-case-sensivity (or s.th. similar). 2.) Being real-world compliant: By default postfwd uses the same counter for bob@alice.local and BoB@alice.local. To change this the admin has to specify --strict-rfc5321-rate-case-sensivity. What's your opinion on that? Do you know any real-world systems that distinguish between bob@alice.local and BoB@alice.local (means that both point to different mailboxes). Thank you in advance Jan
Re: Case sensivity: Strict rfc5321 or reality compliance
Am 15.04.2013 14:24, schrieb Jan P. Kessler: Hi, sorry, I know this is not directly related to postfix but I know that there are several very experienced people reading this list. My question is how you (the people that use and administer mailservers) handle the localpart case sensivity according to rfc5321: The local-part of a mailbox MUST BE treated as case sensitive. nobody i see all the time users having their adress mixed uppercase and lowercase from incoming mails and i would call it quite dumb if m...@dydomain.tld and m...@dydomain.tld is received from different people signature.asc Description: OpenPGP digital signature
Re: Case sensivity: Strict rfc5321 or reality compliance
On 4/15/2013 8:24 AM, Jan P. Kessler wrote: Hi, sorry, I know this is not directly related to postfix but I know that there are several very experienced people reading this list. My question is how you (the people that use and administer mailservers) handle the localpart case sensivity according to rfc5321: The local-part of a mailbox MUST BE treated as case sensitive. You are misunderstanding. Relaying MTAs MUST treat the local-part as case sensitive. IOW, until the message is received at the destination, case must be preserved. However, the RFC does NOT require any organization to treat their local addresses as case sensitive. It would be pointless anyway as you could just say all the variations of case are aliases.
Re: Case sensivity: Strict rfc5321 or reality compliance
localpart case sensivity according to rfc5321: The local-part of a mailbox MUST BE treated as case sensitive. You are misunderstanding. Relaying MTAs MUST treat the local-part as case sensitive. IOW, until the message is received at the destination, case must be preserved. However, the RFC does NOT require any organization to treat their local addresses as case sensitive. It would be pointless anyway as you could just say all the variations of case are aliases. Really? I thought about that, but I think it's not that easy. What if you are a provider (relaying for one or more organisations) and the rate-limiting happens at your relay? I know about several providers using rate limits to throttle their customers on unusual mass-mailing events. Of course these rate limits will not modify the envelope address case but nevertheless have consequences depending on their implementation (means if you count bob and BoB differently or not). Again sorry to the list maintainers. If you think, that this is not the right place for this discussion, anybody is free to share his opinion at info at postfwd dot org.
Re: Case sensivity: Strict rfc5321 or reality compliance
On 4/15/2013 9:44 AM, Jan P. Kessler wrote: localpart case sensivity according to rfc5321: The local-part of a mailbox MUST BE treated as case sensitive. You are misunderstanding. Relaying MTAs MUST treat the local-part as case sensitive. IOW, until the message is received at the destination, case must be preserved. However, the RFC does NOT require any organization to treat their local addresses as case sensitive. It would be pointless anyway as you could just say all the variations of case are aliases. Really? I thought about that, but I think it's not that easy. What if you are a provider (relaying for one or more organisations) and the rate-limiting happens at your relay? I know about several providers using rate limits to throttle their customers on unusual mass-mailing events. Of course these rate limits will not modify the envelope address case but nevertheless have consequences depending on their implementation (means if you count bob and BoB differently or not). Again sorry to the list maintainers. If you think, that this is not the right place for this discussion, anybody is free to share his opinion at info at postfwd dot org. Taken strictly, as not being the destination host, the relay would need to treat the addresses as case-sensitive, at least for relaying purposes. That said, rate limiting in and of itself would not be affected by the RFC. IOW, the RFC has absolutely nothing to say about the matter. Basically, all the RFC is saying is that a relay cannot assume addresses are case insensitive and MUST preserve the case of the address in the envelope. It has no bearing on anything else.
Re: R: Scheduling policies for outgoing smtp server
Coming back to original example of a one-million message queue: Postfix is designed to survive extreme overload, but all mail will be delayed. This is no different than the road to the airport: when it becomes full, all vehicles will be delayed. Both the Postfix scheduler and the road to the airport have a finite capacity. Once they become congested you get first-in, first-out. If you want fairness with 1M+ messages and a scheduler with a fixed memory budget, then you need a scheduler admission policy. For the problem at hand, the important scheduler decisions are: a) Output side: which (destination, recipients) to deliver next. I think that this part does not need to be changed, precisely because the scheduler can see only a subset of all recipients in the mail queue. The trick is deciding what recipient subset the scheduler gets to see. b) Input side: how many recipients to read from a queue file. This is an important part of Patrik's scheduler, but fairness between multi- and single-recipient mail is not the issue here. In the case of a queue full of single-recipient messages, the only choice is to exclude a queue file from consideration (in terms of the road to the airport, to not allow a car to enter the road). Basically this means some sort of active queue admission policy that includes queue file skipping based on some policy. A queue file can be skipped in more than one way. Besides moving a file the deferred queue, a file can also be skipped by leaving it in the incoming queue and setting the mtime time stamp a little into the future. We could also add an overflow queue that is read when the incoming queue becomes empty. Which option is better depends on how large the amount excess of mail is. Wietse
Re: Case sensivity: Strict rfc5321 or reality compliance
Jan P. Kessler: Hi, sorry, I know this is not directly related to postfix but I know that there are several very experienced people reading this list. My question is how you (the people that use and administer mailservers) handle the localpart case sensivity according to rfc5321: The local-part of a mailbox MUST BE treated as case sensitive. This is about preserving addresses case. That is important for transit email, but it has nothing to do with access policy. When Postfix is final destination, whether case is preserved depends the message store. For example, delivery to UNIX system accounts is case-insensitive with respect to the envelope recipient. For rate limits and other policies it makes a lot of sense to keep them case-insensitive. Wietse
Re: Case sensivity: Strict rfc5321 or reality compliance
On 4/15/2013 7:24 AM, Jan P. Kessler wrote: Hi, sorry, I know this is not directly related to postfix but I know that there are several very experienced people reading this list. My question is how you (the people that use and administer mailservers) handle the localpart case sensivity according to rfc5321: The local-part of a mailbox MUST BE treated as case sensitive. Background ist the development of postfwd rate limiting, which allows amongst others to place maximum values (count or volume) for sender or recipient addresses. To do so postfwd creates a hash with the given address as index. At the moment it is easy to trick such limits by varying the case of the addresses, meaning that postfwd will create different counters for bob@alice.local, BoB@alice.local and bob@AlicE.local. No question - with future versions postfwd will definitely treat the domainpart case-insensitive, so that bob@alice.local and bob@AlicE.local will have the same counter. But for the localpart I see two options, wondering which I should choose: 1.) Being strictly rfc5321 compliant: By default postfwd creates two different counters for bob@alice.local and BoB@alice.local. To change this the admin has to specify --non-rfc5321-rate-case-sensivity (or s.th. similar). 2.) Being real-world compliant: By default postfwd uses the same counter for bob@alice.local and BoB@alice.local. To change this the admin has to specify --strict-rfc5321-rate-case-sensivity. What's your opinion on that? Do you know any real-world systems that distinguish between bob@alice.local and BoB@alice.local (means that both point to different mailboxes). Thank you in advance Jan I believe the RFC is mostly saying that MTAs must not alter the localpart case prior to final delivery, which postfwd cannot do regardless. As for a sane default, case-insensitive makes the most sense as that will be the overwhelming majority of end users. The option to enable case-sensitive operation should probably be a per-domain property. But almost no one will use it. Several years ago I worked with some windows-based mail system that had case-sensitive localparts, so they do -- or did -- exist. Fortunately that customer just used all-lowercase usernames. (Their users were constantly getting failed mail because they forgot to lower-case the username. I can't imagine anyone doing this on purpose.) -- Noel Jones
Re: Stripping Received: headers
On Fri, 12 Apr 2013, Wietse Venema wrote: Geoff Shang: On Fri, 12 Apr 2013, Wietse Venema wrote: You need to do postfix reload after editing master.cf. I did. I did it again for good measure - no difference. Are you using receive_override_options? in main.cf or master.cf? No. Geoff.
Re: File descriptor issue in Solaris 11.1 ?
Jaco Lesch: Since we have upgraded to Solaris 11.1 from Solaris 11viathe IPS repository, we are observing the following in our Postfix smtp mail relay logs: Apr 15 13:27:34 smtp-server postfix/smtp[25213]: [ID 947731 mail.crit] fatal: stream_connect: send file descriptor: Resource temporarily unavailable Options: - Get a faster machine, more memory, or tune the kernel. - Install Postfix 2.10 (this no longer uses Solaris streams). - Disable Postfix connection caching. /etc/postfix/main.cf: smtp_connection_cache_on_demand = no smtp_connection_cache_destinations = Wietse
Re: SMTPS 465
On 4/15/2013 6:57 AM, Joan Moreau wrote: Reverted to 3.7.10. Recompiled openssl + cyrus + posfix . Same errors. Where does the inconsistency reside ? You will probably not get the answer from the Postfix mailing list, as this is not a problem with Postfix, and it appears that nobody here is willing to dedicate additional time to helping you debug/fix a non-Postfix problem. 2013-04-15T13:55:29.921960+02:00 server postfix/smtpd[3308]: warning: TLS library problem: 3308:error:1411C146:SSL routines:tls1_prf:unsupported digest type:t1_enc.c:276: 2013-04-15T13:55:29.921966+02:00 server postfix/smtpd[3308]: warning: TLS library problem: 3308:error:140D308A:SSL routines:TLS1_SETUP_KEY_BLOCK:cipher or hash unavailable:t1_enc.c:597: I think it's time for you to move (back?) into the distribution fold, to discontinue building your userland from scratch. You are an end user, not a developer. The situation you find yourself in is the exact reason why the first Linux distributions were created, and still exist today. Which is to get a pre built system where the kernel and all of the package and library versions work together, without end user debugging required. In fact, you are the current poster child for the Linux distribution model. -- Stan
Re: R: Scheduling policies for outgoing smtp server
On Mon, Apr 15, 2013 at 01:20:58PM +0200, Giorgio Luchi wrote: How can we implement round-robin by sender ip/authenticated user and to preserve the memory constraint too? - sender is the sender's ip address or the authenticated user name (i.e. 80.93.143.50 or giorgio.luchi) - rrsender_message_limit is the max number of messages in the sender queue (i.e. 3) - rrsender_queue_limit is the max number of senders actually loaded in RAM (i.e. 6000) [...] I hope this can help in understanding and in finding a solution to what we need. Your specific proposal is not viable. The scheduler (queue manager processing the active queue) works with a bounded subset of messages and message recipients. Think of these as people who are in the airport terminal whose ticket class determines which line they may join, with some lines getting better service than others. The capacity of the capacity of the roads leading to the terminal is finite. A first-class ticket holder cannot take advantage of preferrential treatment at the terminal if he is stuck behind thousands of economy ticket holders on a congested highway. The highway is analogous to the Postfix incoming and deferred queues and finite capacity of smtpd(8) processes to take in new mail. You must abandon all hope of trying to make the highway more fair than FIFO, this is not possible. What is possible is tweaking the algorithm at the terminal. It is in principle possible to adjust the scheduler algorithm for selecting the next message to send (out of those already in the active queue). Right now the scheduler is able to interleave delivery of later arriving small messages with progressive delivery of larger (mailing list) messages, so that a single newsletter delivery does not substantially delay the delivery of all subsequent mail. If the same newsletter were to be injected into the queue as as a burst of individual messages the scheduler would not be able to apply Patrick Raq's nqmgr FIFO preemption logic. One could conceivable enhance the algorithm to support a notion of job groups, where the pre-emption logic works at two different levels. - A job group can be preempted by a later job group after the first job group consumes enough delivery slots. - A job within a group can be preempted by a later job within that group after the first job consumes enough delivery slots. It would then be up to the input stage to tag queue files with job group identifiers. This could be done a policy service that returns a new access(5) action to set the job group id. The effect is to logically re-assemble a multi-message bulk mailing from a single source (be it by IP, SASL username, sender address, ...) into a single logical message which is subject to the scheduler preemption algorithm. This is perhaps a fruitful direction, though an ISP would likely get more bang for the buck by rate limiting input with a policy service. The outlined design would be of greater utility in large corporate and some hosted email marketing scenarios. If anyone were to design an build such a thing on contract, it would probably be Patrick Raq (if he's available). Otherwise, nobody else comes to mind, you'd have to hire someone who's capable of understanding Patrick's design in detail, and extending it correctly. The result would be unlikely to be incorporated into the mainstream Postfix release unless it were exceedingly well documented and implemented with care. Most likely you'd be stuck maintaining it as a private patch indefinitely. So the best approach is perhaps to find some other MTA that does what you want if one exists. Otherwise, try to leverage the existing feature set to approximate what you want, rate limiting input is my best suggestion. -- Viktor.
Postscreen and Memcached
Hello, I have postscreen working fine, on localhost, with memcached though am getting confused on how one sets it up for multi-server usage ? We would like out two front-end MTAs to share any IPs that have been added to the cache. How would we achieve this please ? Many thanks.
Re: Scheduling policies for outgoing smtp server
Am 2013-04-05 12:36, schrieb Giorgio Luchi: - User A [...] sends 1 different email to 1 million of different domain destinations [...] - User B [...] sends an email to a different more domain destinatio Doesn't this scream for two different postfix instances? - One high priority instance with strict rate limiting - One low priority instance for bulk messages, possibly with some sort of traffic shaping You could check for a Precedence: bulk header to forward mails to the correct postfix instance automatically. Or am I overlooking an obvious problem here? Timo
Re: Postscreen and Memcached
Phil Daws: I have postscreen working fine, on localhost, with memcached though am getting confused on how one sets it up for multi-server usage ? We would like out two front-end MTAs to share any IPs that have been added to the cache. How would we achieve this please ? postscreen requires a *persistent* database. memcached can be used on top of a *persistent* database. memcached can be shared between between different hosts, but that is not enough. You need to share the persistent cache, too. postscreen does not support a *persistent* database that can be shared between different hosts. I haven't tried the Postfix memcache client against memcachedb. This might work, but it may not be fast enough. Wietse
sender_dependent_relayhost_maps (fwd)
Dear Gentle Folk, I wish to relay all mail from our outgoing mail server, smtp.lightlink.com, through a default outgoing barracuda, EXCEPT mails FROM specified addresses that wish to do large authorized mailings that tend to bury the barracuda. These exceptions should be delivered directly by the outgoing mail server, smtp.lightlink.com I have #relayhost = sender_dependent_relayhost_maps = hash:/etc/postfix/relayhost relayhost file says * [barracuda.lightlink.com]:25 frien...@spammer.com : the first line alone doesn't work, the second line doesn't either :) I apologize for my stupidity in advance. Thanks Homer Homer Wilson Smith Clean Air, Clear Water,Art Matrix - Lightlink (607) 277-0959 A Green Earth, and Peace, Internet, Ithaca NY ho...@lightlink.com Is that too much to ask? http://www.lightlink.com
Re: sender_dependent_relayhost_maps (fwd)
Homer Wilson Smith: #relayhost = sender_dependent_relayhost_maps = hash:/etc/postfix/relayhost relayhost file says * [barracuda.lightlink.com]:25 frien...@spammer.com : I don't know what * or : is supposed to mean. They certainly are not documented for sender_dependent_relayhost_maps. What the documentation does support is: /etc/postfix/main.cf: relayhost = [barracuda.lightlink.com]:25 sender_dependent_relayhost_maps = hash:/etc/postfix/relayhost /etc/postfix/relayhost frien...@spammer.com smtp.lightlink.com Don't forget postmap hash:/etc/postfix/relayhost. As documented sender_dependent_relayhost_maps takes precedence over relayhost. Wietse
Re: Case sensivity: Strict rfc5321 or reality compliance
Thanks to anybody for sharing your opinions and thoughts. I decided that the default rate limit functions will operate completely case-insensitive, because this seems to be what people (including me) expect. As most people also seem to have expected that behaviour in the past, I think this is one of the seldom cases where it makes sense to 'break' backwards compatibility. Additionally I will add rate() functions with different names that treat the local part case-sensitive, so that there will be an option to control the behaviour per domain/address (or even other criteria) for those who need it (condolences). Again, thank you. Jan
Re: sender_dependent_relayhost_maps (fwd)
On Mon, Apr 15, 2013 at 04:17:23PM -0400, Homer Wilson Smith wrote:
Dear Gentle Folk,
I wish to relay all mail from our outgoing mail server,
smtp.lightlink.com, through a default outgoing barracuda, EXCEPT
mails FROM specified addresses that wish to do large authorized
mailings that tend to bury the barracuda. These exceptions should
be delivered directly by the outgoing mail server,
smtp.lightlink.com
I have
sender_dependent_relayhost_maps = hash:/etc/postfix/relayhost
Instead use:
main.cf:
# If you have cdb, adjust accordingly if not
default_database_type = cdb
indexed = ${default_database_type}:${config_directory}/
sender_dependent_default_transport_maps = ${indexed}sender-transport
# Default transport for all senders, includes the default relay
default_transport = smtp:[relay.example.com]
# Empty default relayhost!
relayhost =
sender-transport:
# The smtp transport with no nexthop sends direct, since relayhost
# is not set. The senders below bypass the default transport which
# specifies the normal relay
bu...@example.com smtp
bu...@example.com smtp
...
Don't forget to postmap the sender-transport file:
Makefile:
all: sender-transport.cdb
sender_transport.cdb: sender-transport
postmap cdb:sender-transport
--
Viktor.
Re: Postscreen and Memcached
On 4/15/2013 1:59 PM, Wietse Venema wrote: Phil Daws: I have postscreen working fine, on localhost, with memcached though am getting confused on how one sets it up for multi-server usage ? We would like out two front-end MTAs to share any IPs that have been added to the cache. How would we achieve this please ? postscreen requires a *persistent* database. memcached can be used on top of a *persistent* database. memcached can be shared between between different hosts, but that is not enough. You need to share the persistent cache, too. postscreen does not support a *persistent* database that can be shared between different hosts. I haven't tried the Postfix memcache client against memcachedb. This might work, but it may not be fast enough. Wietse, I recall being part of a discussion quite some time ago, and I thought this had been worked out already, using proxymap. My recollection is really vague. What am I remembering, if not postscreen cache sharing between hosts? -- Stan
