Re: Auto-whitelist recipients
Am 04.09.2012 08:37, schrieb Robert Schetterer: > Am 03.09.2012 20:36, schrieb Eddy Ilg: >> Dear Postfix List, >> >> >> I'd like to continously update whitelist for spamassassin of recipients >> that my sasl users have sent mail to (i.e. when the recipients reply >> they will surely not be considered as spam). I am not using per-user >> spamassassin configurations (only a global configuration). > > the problem will be that you have to restart spamassassin > each time you whitelist if using spamd with local files and global setup > ( dont know it might better with some sql setup ), i use spamass-milter > and dont scan for spam with sasl authed users, in addition i use > clamav-milter with sanesecurtiy antispam stuff > for all users, this isnt ideal but its fast enough and havent got any > problems since years > > anyway you may use amavis, and ask on their list, perhaps its more easy > to goal what you want with some amavis setup >> >> I've found several approaches but none seems to fit... Where is the best >> place to insert a script that grabs recipient of mails being sent by >> sasl-authenticated users? > > for a general "whitelist" feature you may use i.e > > http://mailfud.org/postpals/ >> >> >> Best regards, >> >> >> Eddy > > i forgot , you can always write a log analyse script for your sasl authed users recipients, i.e with bash and awk running at pre stage on daily logrotate which does whitelist and restart spamassassin, i dont would recommand that you may have tons of whitelistings after time, and the senders could ever be faked some day so you will lost chance to scan them then however, perhaps do whitelisting yourself for well knowed used domains by your users after log analyse -- Best Regards MfG Robert Schetterer
Re: Auto-whitelist recipients
Am 03.09.2012 20:36, schrieb Eddy Ilg: > Dear Postfix List, > > > I'd like to continously update whitelist for spamassassin of recipients > that my sasl users have sent mail to (i.e. when the recipients reply > they will surely not be considered as spam). I am not using per-user > spamassassin configurations (only a global configuration). the problem will be that you have to restart spamassassin each time you whitelist if using spamd with local files and global setup ( dont know it might better with some sql setup ), i use spamass-milter and dont scan for spam with sasl authed users, in addition i use clamav-milter with sanesecurtiy antispam stuff for all users, this isnt ideal but its fast enough and havent got any problems since years anyway you may use amavis, and ask on their list, perhaps its more easy to goal what you want with some amavis setup > > I've found several approaches but none seems to fit... Where is the best > place to insert a script that grabs recipient of mails being sent by > sasl-authenticated users? for a general "whitelist" feature you may use i.e http://mailfud.org/postpals/ > > > Best regards, > > > Eddy -- Best Regards MfG Robert Schetterer
Re: Bulk Mailing Performance
On 9/2/2012 11:14 AM, Sam Jones wrote: On Sun, 2012-09-02 at 15:39 +, Viktor Dukhovni wrote: On Sun, Sep 02, 2012 at 10:43:07AM +0100, Sam Jones wrote: More to satisfy my own curiosity than anything else, I'm wondering about the performance that could be squeezed out of Postfix in a bulk mailing capacity. Running a high volume bulk email platform is not a software problem. It is a logistics problem. Enrolling on the whitelists and feedback loops of various large email providers, handling bounce-backs, jumping through rate-limit hoops, ... [...] I guess what I'm querying in a way is some of the sales blurb from people like PowerMTA & GreenArrow and the remarks they make about open source solutions like Postfix etc. This one in particular: "Open source Mail Transfer Agents (MTAs) often max out between 20 and 30 thousand messages per hour. GreenArrow can send 300,000 messages per hour—more than ten times as fast." Knowing absolutely nothing about the software mentioned - I would say there is a difference between messages SENT vs messages DELIVERED. I realize many will immediately correct me and say even Postfix can't guarantee delivery to a given recipient - merely acknowledgement of the recipient server's acceptance - but I don't know how else to discriminate between a single-pass of a message, without retries, without verification, without greylist tolerance, without reporting, just knock on the door and try to shove it on - vs reliable message handling. Again, knowing nothing about alternatives to Postfix - I question whether software intended for bulk mailing purposes is designed in such a manner. As a crude analogy, even the best machine gun doesn't have a fraction of the accuracy of a quality sniper rifle - but on the other hand a machine gun will put a lot more lead downrange. Different tools for different purposes. Spray-and-Pray - or deliver the personal message. -- Daniel
Re: temporarily suspending delivery
On Mon, 2012-09-03 at 19:36:46 -0400, b...@bitrate.net wrote: > i have an mx which then subsequently delivers incoming mail from the > internet to another computer [ via relay_transport = > relay-mda:[mda.example.com]:smtp-relay ] for further processing. > while performing some maintenance on mda.example.com, i'd like to > configure postfix on the mx to accept all mail as it has been, but > instead of then delivering to mda.example.com, retain all mail until > it is manually released. it looks like the hold queue may be > appropriate for this? how can i accomplish this? Rather than the hold queue, use the retry service. /path/to/main.cf: transport_maps = hash:/path/to/transport /path/to/transport: mda.example.com retry:4.2.1 mda.example.com is temporarily disabled -- Sahil Tandon
Re: The Yahoo trickle
On 9/3/2012 5:44 PM, Joey Prestia wrote: > On 9/3/2012 10:43 AM, Viktor Dukhovni wrote: >> Sadly, Yahoo discriminates the Postfix connection cache which limits >> connection re-use by time rather than delivery count. Limiting by >> delivery count behaves poorly when one or more of the MX hosts for >> a site is slower than the rest, it becomes a connection "attractor", >> so Postfix uses a better strategy. > I thought we were making good use of our connection caching? Connection caching needs two good dance partners to work effectively. In this case Yahoo is stepping on Postfix' toes. -- Stan
Re: Backup MXs and databases
Am 04.09.2012 01:15, schrieb Titanus Eramius: >> where do you see a risk of silent data corruption? >> >> if this would be the case it would be simply >> impossible have a omplete dbmail-database running >> on a replication salve over 3 years with a lot of >> foreign constraints and a major scheme update >> >> there is NO silent corruption >> please do not post FUD > > I'm somewhat sorry for being unable to ask my question in a manner that > can't be misunderstood, but I suppose it's always possible to find > someting negative if you are looking for it. that is not the point > Like I said "I've been reading up on the subject, but seems to lack the > experince ..." which should be understood as I don't know anything > about the subject besides what I have read. > > Like this > http://www.iheavy.com/2012/04/26/bulletproofing-mysql-replications-with-checksums/ * mixed transactional and non-transactional tables not relevant in this context why would someone mix innodb/myisam a database and transaction? * use of non-deterministic functions such as uuid() not relevant in this context * stored procedures and functions not relevant in this context * update with LIMIT clause not relevant in this context even if, combined with a clear "order by" no problem > So I'm sorry, I don't see the FUD, and because I know next to nothing > about databases, I simply can not see these replication errors as > anything else than corruption*. But please enlighten me, that was why I > posted to the list. for postfix lookup tables you have usually a very simple database scheme with very few changes and 99.9% of all queries are readonly because postfix does even not need any write permissions to the database (and does not have it in any of my setups) so you have a simple webinterface for updates or if you have only a few domains/users maybe phpMyAdmin or terminal would be enough so there is virtually zero danger for get out of sync signature.asc Description: OpenPGP digital signature
temporarily suspending delivery
hi- i have an mx which then subsequently delivers incoming mail from the internet to another computer [ via relay_transport = relay-mda:[mda.example.com]:smtp-relay ] for further processing. while performing some maintenance on mda.example.com, i'd like to configure postfix on the mx to accept all mail as it has been, but instead of then delivering to mda.example.com, retain all mail until it is manually released. it looks like the hold queue may be appropriate for this? how can i accomplish this? thanks -ben
Re: Backup MXs and databases
On Tue, 04 Sep 2012 00:39:08 +0200 Reindl Harald wrote: > Am 03.09.2012 23:56, schrieb Titanus Eramius: > > > > MySQL Replication, which seems a bit dodgy, with the risk of silent > > data corruption. > > where do you see a risk of silent data corruption? > > if htis would be the case it would be simply > impossible have a omplete dbmail-database running > on a replication salve over 3 years with a lot of > foreign constraints and a major scheme update > > there is BO silent corruption > please do not post FUD I'm somewhat sorry for being unable to ask my question in a manner that can't be misunderstood, but I suppose it's always possible to find someting negative if you are looking for it. Like I said "I've been reading up on the subject, but seems to lack the experince ..." which should be understood as I don't know anything about the subject besides what I have read. Like this http://www.iheavy.com/2012/04/26/bulletproofing-mysql-replications-with-checksums/ And this http://www.xaprb.com/blog/2007/11/08/how-mysql-replication-got-out-of-sync/ (which I found here http://forums.mysql.com/read.php?27,216438,216438 ) And so on http://www.pythian.com/news/1273/mysql-replication-failures/ So I'm sorry, I don't see the FUD, and because I know next to nothing about databases, I simply can not see these replication errors as anything else than corruption*. But please enlighten me, that was why I posted to the list. --- * "Data corruption refers to errors in computer data that occur during writing, reading, storage, transmission, or processing, which introduce unintended changes to the original data." https://en.wikipedia.org/wiki/Data_corruption
Re: Backup MXs and databases
On Mon, 2012-09-03 at 23:56 +0200, Titanus Eramius wrote: > So, I guess my question is: How do you, good and experienced folks, > keep your backup MXs updated? > > I've looked at two solutions so far: > MySQL Replication +1 for mysql replication for backup mail servers. It's been pretty reliable for me. As long as you run some sort of regular process to check that the databases are in sync then you'll be fine (see maatkit). Andy
Re: The Yahoo trickle
On 9/3/2012 10:43 AM, Viktor Dukhovni wrote: > On Sun, Sep 02, 2012 at 08:07:21PM -0700, Joey Prestia wrote: > >> yahoo_destination_concurrency_limit = 20 > > This setting is trumpted by the setting below: > >> yahoo_destination_rate_delay = 1s > > You have serialized deliveries to Yahoo, they happen one at a time. > Given that each delivery takes ~5s, there is not much point in > doing that, you can instead set a low concurrency, and get a bunch > more throughput by not setting an explicit rate limit. (perhaps > 2 deliveries per second with a concurrency of 10, rather than 1 > delivery every 5 seconds). So what I would need then is in main.cf and in master.cf in my transport would be this? yahoo_destination_concurrency_limit = 10 -o smtp_connection_cache_on_demand=no > > For more throughput, you need more IP addresses, perhaps even in > distint address blocks, ... > > Sadly, Yahoo discriminates the Postfix connection cache which limits > connection re-use by time rather than delivery count. Limiting by > delivery count behaves poorly when one or more of the MX hosts for > a site is slower than the rest, it becomes a connection "attractor", > so Postfix uses a better strategy. > I thought we were making good use of our connection caching? > You can just disable connection caching with Yahoo, they rarely > have unreachable MX hosts, your deliveries are just as slow whether > connections are cached or not. >
Re: Backup MXs and databases
Am 03.09.2012 23:56, schrieb Titanus Eramius: > Hello good folks > > I have recently brought my very first mailserver online, and have been > testing it for the past month or so. Since the setup needs to be > redundant, I have also brought a secondary mailserver online on it's own > domain, and everything seems to run smoothly. > > MySQL Replication, which seems a bit dodgy, with the risk of silent > data corruption. where do you see a risk of silent data corruption? if htis would be the case it would be simply impossible have a omplete dbmail-database running on a replication salve over 3 years with a lot of foreign constraints and a major scheme update there is BO silent corruption please do not post FUD signature.asc Description: OpenPGP digital signature
Backup MXs and databases
Hello good folks I have recently brought my very first mailserver online, and have been testing it for the past month or so. Since the setup needs to be redundant, I have also brought a secondary mailserver online on it's own domain, and everything seems to run smoothly. It's a Debian, Postfix, Dovecot, postfixAdmin, SQLGrey and Squrriel Mail setup, with virtual users only, and MySQL as the central component. The database is what my question is about. As far as I know, the best way to fight of spam and backscatter, is if the backup MX uses the same database as the primary. I've been reading up on the subject, but seems to lack the experince to take a dissicion on what way to keep the backup MX database updated. It does not need to be real-time, or anywhere close to it. For this setup twice a day will probably be fine. So, I guess my question is: How do you, good and experienced folks, keep your backup MXs updated? I've looked at two solutions so far: MySQL Replication, which seems a bit dodgy, with the risk of silent data corruption. Bash scipting with rsync could do the job, but seems like a less clean solution. Thank you for your time
Auto-whitelist recipients
Dear Postfix List, I'd like to continously update whitelist for spamassassin of recipients that my sasl users have sent mail to (i.e. when the recipients reply they will surely not be considered as spam). I am not using per-user spamassassin configurations (only a global configuration). I've found several approaches but none seems to fit... Where is the best place to insert a script that grabs recipient of mails being sent by sasl-authenticated users? Best regards, Eddy
Re: The Yahoo trickle
On Sun, Sep 02, 2012 at 08:07:21PM -0700, Joey Prestia wrote: > yahoo_destination_concurrency_limit = 20 This setting is trumpted by the setting below: > yahoo_destination_rate_delay = 1s You have serialized deliveries to Yahoo, they happen one at a time. Given that each delivery takes ~5s, there is not much point in doing that, you can instead set a low concurrency, and get a bunch more throughput by not setting an explicit rate limit. (perhaps 2 deliveries per second with a concurrency of 10, rather than 1 delivery every 5 seconds). For more throughput, you need more IP addresses, perhaps even in distint address blocks, ... Sadly, Yahoo discriminates the Postfix connection cache which limits connection re-use by time rather than delivery count. Limiting by delivery count behaves poorly when one or more of the MX hosts for a site is slower than the rest, it becomes a connection "attractor", so Postfix uses a better strategy. You can just disable connection caching with Yahoo, they rarely have unreachable MX hosts, your deliveries are just as slow whether connections are cached or not. -- Viktor.
Re: Bulk Mailing Performance
On Sun, 2012-09-02 at 22:46 +0200, Lorens Kockum wrote: > The exact same question was sent by someone calling himself > "Ron White" to the exim mailing list at almost exactly the same > time. Peddling one's services by soliciting comparisons with > competitors is so passé . . . > Yes, it was. Well done. The question applied to both MTA's and funny enough, the use of Aliases on the internet is nothing new. Thanks to those that contributed useful information. I think it's safe to say that the sales blurb is looking at a very basic scenario.
Re: headers_check confusion
an...@isac.gov.in: > Dear List, > > I have following header_check > > /^X-ABC:.*XYZ.*/ DUNNO This matches X-ABC: followed by whatever, followed by XYZ. > !/^X-ABC:.*XYZ.*/ FILTER smtp:a.b.c.d: This matches ALL OTHER MESSAGE HEADERS. Wietse
Re: Bulk Mailing Performance
DTNX Postmaster wrote: They aren't my perfect world criteria, but a direct quote from Sam Jones' earlier buzzword compliant reply. It was meant to illustrate the often ridiculous nature of vendor benchmarks, how useless they are in real world situations, and therefore how silly it is to pick software based on theoretical limits you will most likely never hit. Not really ridiculous. All those benchmarks are interesting, as they represent, say, the "intrinsic performance of the software". The problem is to tell (for the vendor) and to take into account (for the reader) the conditions at which the benchmark was done. But, sure, two pieces of software can be compared only if measurings are done with the same conditions. And one software which has better "intrinsic performance" may not be better in real world conditions. --
Re: The Yahoo trickle
On 9/3/2012 3:50 AM, Stan Hoeppner wrote: > On 9/2/2012 10:07 PM, Joey Prestia wrote: >> Hi all, >> >> I am familiar with yahoo being difficult to send email to > > [snip] > >> Can anyone offer any guidance on what direction I need to go? > > Start here: > http://help.yahoo.com/l/us/yahoo/mail/postmaster/bulkv2.html > Hi Stan, I did that some time ago. I am white listed with them and have published SPF Records and use DKIM to sign all outgoing mail. I am not receiving any error codes only (250 ok dirdel) in my logs. I will recheck with yahoo on my bulk sending status and validate it indeed as it should be. Joey
headers_check confusion
Dear List, I have following header_check /^X-ABC:.*XYZ.*/ DUNNO !/^X-ABC:.*XYZ.*/ FILTER smtp:a.b.c.d: I tried these header checks with warn, and only one is getting matched based on headers which I send. But, what I find is, when actual mail is sent, always second header gets activated. Even I tried to change the order of header_checks, same result. Basically what I want is, if header X-ABC matches, it should deliver the mail within the same postfix instance, and if it does not match, it should relay to a.b.c.d Where I am going wrong? Regards, Anant. -- Confidentiality Notice: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. --
Re: Bulk Mailing Performance
On Sep 3, 2012, at 13:05, Stan Hoeppner wrote: > On 9/3/2012 12:02 AM, DTNX Postmaster wrote: > >> In other words, if 'we strip this back to hypothetical and assume a >> perfect world without any issues', this 'GreenArrow' maxes out at >> 300,000 messages per hour. Postfix can send 10,8 million messages per >> hour, more than 35 times as fast*. > > In all fairness, given your "perfect world" criteria, this ESP would be > moving a lot more mail as well, with no restrictions on the outbound > pipe or at the receiver. > > But as others have correctly pointed out, the issue here isn't MTA > performance, it's administrative performance. The last thread I > responded to demonstrates this. The big advantage ESPs have is their > established relationships with the freemailers and other large mailbox > providers. These allow them greater throughput than the unwashed bulk > sender, at least into the receiver's initial queue. They aren't my perfect world criteria, but a direct quote from Sam Jones' earlier buzzword compliant reply. It was meant to illustrate the often ridiculous nature of vendor benchmarks, how useless they are in real world situations, and therefore how silly it is to pick software based on theoretical limits you will most likely never hit. Not enough sarcasm, I guess ;-) Cya, Jona
Re: Bulk Mailing Performance
On 9/3/2012 12:02 AM, DTNX Postmaster wrote: > In other words, if 'we strip this back to hypothetical and assume a > perfect world without any issues', this 'GreenArrow' maxes out at > 300,000 messages per hour. Postfix can send 10,8 million messages per > hour, more than 35 times as fast*. In all fairness, given your "perfect world" criteria, this ESP would be moving a lot more mail as well, with no restrictions on the outbound pipe or at the receiver. But as others have correctly pointed out, the issue here isn't MTA performance, it's administrative performance. The last thread I responded to demonstrates this. The big advantage ESPs have is their established relationships with the freemailers and other large mailbox providers. These allow them greater throughput than the unwashed bulk sender, at least into the receiver's initial queue. -- Stan
Re: The Yahoo trickle
On 9/2/2012 10:07 PM, Joey Prestia wrote: > Hi all, > > I am familiar with yahoo being difficult to send email to [snip] > Can anyone offer any guidance on what direction I need to go? Start here: http://help.yahoo.com/l/us/yahoo/mail/postmaster/bulkv2.html -- Stan
Re: SMTP authentication issue
On Mon, Sep 03, 2012 at 10:28:06AM +0200, Helga Mayer wrote:
[...]
> >user name jhondoe
> >password 12345678
> >
> >but when user authenticate 12345678__-- authenticate again.
> >
> >but when users enter a12345678 can't authenticate
> The first 8 characters matter. This looks like a problem of the backend.
> Though I have never heard that openldap restricts the password to 8
> characters.
> To my experience solaris does.
Is it possible that LDAP contains DES ({crypt}) encrypted password? Then
only the first 8 characters of the password counts, AFAIK. It's the
limitation of the choosen password hash algorithm, one should select another
one (also recommended because of the weakness of DES nowdays). IMHO.
Re: SMTP authentication issue
On 09/03/2012 09:51 AM, Selcuk Yazar wrote: Hi, we have weird issue on postfix smtp authenticaion(postfix-openldap-dovecot). one user enter wrong characters after his correct password authentication again. i mean user name jhondoe password 12345678 but when user authenticate 12345678__-- authenticate again. but when users enter a12345678 can't authenticate The first 8 characters matter. This looks like a problem of the backend. Though I have never heard that openldap restricts the password to 8 characters. To my experience solaris does. Regards Helga Mayer
Re: Bulk Mailing Performance
* Viktor Dukhovni : > Running a high volume bulk email platform is not a software problem. > It is a logistics problem. Enrolling on the whitelists and feedback > loops of various large email providers, handling bounce-backs, > jumping through rate-limit hoops, ... Absolutely. -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebra...@charite.de | http://www.charite.de
Re: Bulk Mailing Performance
* Sam Jones : > More to satisfy my own curiosity than anything else, I'm wondering about > the performance that could be squeezed out of Postfix in a bulk mailing > capacity. The problem is mostly on the receiving side, when the receiving system starts throtteling you. > I have a client that currently uses and ESP who have an astounding > throughput of up to a million messages per hour. This brought up a > discussion about high-performance MTAs and tuning and the general > comments I'm hearing are that things like Postfix, Exim, Sendmail & > are just not man enough for such a task and the absolute best you could > expect from any of them is about 100k messages per hour. I once sent 2096/min*60min = 125.760mails/minute on mail.python.org and there the generation of the mails is the limiting factor. -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebra...@charite.de | http://www.charite.de
