[qmailtoaster] Re: Sanesecurity, spamassassin spamdyke
On 01/25/2012 09:50 PM, Casey Price wrote: On another note...that link that Eric previously shared from Bill Schupp's site shows spamd running on a separate host with the spamc client running on the inbound boxes. How might one go about setting up something like this, and is it recommended? I believe the reason we had separated out the GW boxes from the SA boxes was because there were times that the GW boxes would get overloaded trying to process messages using spamassassin and we'd end up with a huge queue. So if I'm interpreting this correctly, if we made the SA1 box purely a spamassassin box (which it pretty much is now, but all the mail is being passed from GW1 via smtproutes) and then had spamc running on GW1, that would probably solved some of my problems don't you think? At least the ones I had been having from SaneSecurity and it sending bounces back to my GW box. Having spamd running on a separate host *might* be appropriate with 2 or more gateways, but not with just one. The main reason being that with a separate host, there's no potential performance gain due to i/o caching, which can be substantial. I would wait and see how the single box performs. The stock QMT isn't really tuned at all for major ISP type installations. With a little tuning, QMT can operate at peak capacity while not becoming overloaded. Tuning parameters such as the number of connections and spamc children can do wonders. You might also consider making the /var/qmail/simscan folder a tmpfs, but if the system has ample ram then linux i/o caching can achieve the same result. You can also consider compiling the spamassassin code, although I expect the gains from that aren't significant unless your host is CPU bound. We really need to do some work on documenting tuning best practices, and get this on the wiki. Would someone care to tackle this? In any case, I expect that a single host could handle your load. Besides which, what's so bad about deferring some connections occasionally? So the message sits in the sender's queue a little longer and the message doesn't arrive quite as quickly. I think this is reasonable to expect during peak times. As long as this happens just occasionally and not continually, I doubt your customers would even notice. Did I miss (or forget) it, or have you posted what your hardware is? ;) -- -Eric 'shubes' - Qmailtoaster is sponsored by Vickers Consulting Group (www.vickersconsulting.com) Vickers Consulting Group offers Qmailtoaster support and installations. If you need professional help with your setup, contact them today! - Please visit qmailtoaster.com for the latest news, updates, and packages. To unsubscribe, e-mail: qmailtoaster-list-unsubscr...@qmailtoaster.com For additional commands, e-mail: qmailtoaster-list-h...@qmailtoaster.com
Re: [qmailtoaster] Re: Sanesecurity, spamassassin spamdyke
Casey Price Smile Global Technical Support Submit or check trouble tickets http://billing.smileglobal.com www.smileglobal.com http://www.smileglobal.com Follow us on Twitter https://twitter.com/#%21/SmileInternet Find us on Facebook https://www.facebook.com/smileglobal On 1/26/12 10:06 AM, Eric Shubert wrote: On 01/25/2012 09:50 PM, Casey Price wrote: On another note...that link that Eric previously shared from Bill Schupp's site shows spamd running on a separate host with the spamc client running on the inbound boxes. How might one go about setting up something like this, and is it recommended? I believe the reason we had separated out the GW boxes from the SA boxes was because there were times that the GW boxes would get overloaded trying to process messages using spamassassin and we'd end up with a huge queue. So if I'm interpreting this correctly, if we made the SA1 box purely a spamassassin box (which it pretty much is now, but all the mail is being passed from GW1 via smtproutes) and then had spamc running on GW1, that would probably solved some of my problems don't you think? At least the ones I had been having from SaneSecurity and it sending bounces back to my GW box. Having spamd running on a separate host *might* be appropriate with 2 or more gateways, but not with just one. The main reason being that with a separate host, there's no potential performance gain due to i/o caching, which can be substantial. Well, I have 3 different gateways and two SA boxes. Gateway2 is a QMT xen guest running on a Dell PowerEdge 2650. (I believe this machine has 4 or 5G of RAM with dual Xeon 2.6 or 2.8GHz processors). Gateway3 is a VPS I am leasing from ThrustVPS (damnVPS). Nothing spectacular...but it does the job. I will have to double check on GW1. I know that one of the SA boxes should definitely replace it, because they are more powerful machine. I would wait and see how the single box performs. The stock QMT isn't really tuned at all for major ISP type installations. With a little tuning, QMT can operate at peak capacity while not becoming overloaded. Tuning parameters such as the number of connections and spamc children can do wonders. You might also consider making the /var/qmail/simscan folder a tmpfs, but if the system has ample ram then linux i/o caching can achieve the same result. You can also consider compiling the spamassassin code, although I expect the gains from that aren't significant unless your host is CPU bound. We really need to do some work on documenting tuning best practices, and get this on the wiki. Would someone care to tackle this? In any case, I expect that a single host could handle your load. Besides which, what's so bad about deferring some connections occasionally? So the message sits in the sender's queue a little longer and the message doesn't arrive quite as quickly. I think this is reasonable to expect during peak times. As long as this happens just occasionally and not continually, I doubt your customers would even notice. Did I miss (or forget) it, or have you posted what your hardware is? ;)
[qmailtoaster] Re: Sanesecurity, spamassassin spamdyke
On 01/26/2012 06:34 PM, Casey Price wrote: Well, I have 3 different gateways and two SA boxes. Gateway2 is a QMT xen guest running on a Dell PowerEdge 2650. (I believe this machine has 4 or 5G of RAM with dual Xeon 2.6 or 2.8GHz processors). Gateway3 is a VPS I am leasing from ThrustVPS (damnVPS). Nothing spectacular...but it does the job. I will have to double check on GW1. I know that one of the SA boxes should definitely replace it, because they are more powerful machine. Are there any other guests running along side of GW2? I should think you could get rid of GW3 eventually. What are the specs on the SA boxes? The challenge as I see it will be getting from where you're at to where you want to be with little to no disruption. Do you have domains spread across all 3 GWs presently, or is there some redundancy? Likewise for the SA boxes? It might be simpler to drop off a gateway entirely and put an SA box on the edge, rather than trying to put SA functionality into a GW. Especially if you're going to end up with things on the present SA hosts anyhow. Do you have anything else virtual besides GW1? -- -Eric 'shubes' - Qmailtoaster is sponsored by Vickers Consulting Group (www.vickersconsulting.com) Vickers Consulting Group offers Qmailtoaster support and installations. If you need professional help with your setup, contact them today! - Please visit qmailtoaster.com for the latest news, updates, and packages. To unsubscribe, e-mail: qmailtoaster-list-unsubscr...@qmailtoaster.com For additional commands, e-mail: qmailtoaster-list-h...@qmailtoaster.com
Re: [qmailtoaster] Re: Sanesecurity, spamassassin spamdyke
Casey Price Smile Global Technical Support Submit or check trouble tickets http://billing.smileglobal.com www.smileglobal.com http://www.smileglobal.com Follow us on Twitter https://twitter.com/#%21/SmileInternet Find us on Facebook https://www.facebook.com/smileglobal On 1/26/12 6:31 PM, Eric Shubert wrote: On 01/26/2012 06:34 PM, Casey Price wrote: Well, I have 3 different gateways and two SA boxes. Gateway2 is a QMT xen guest running on a Dell PowerEdge 2650. (I believe this machine has 4 or 5G of RAM with dual Xeon 2.6 or 2.8GHz processors). Gateway3 is a VPS I am leasing from ThrustVPS (damnVPS). Nothing spectacular...but it does the job. I will have to double check on GW1. I know that one of the SA boxes should definitely replace it, because they are more powerful machine. Are there any other guests running along side of GW2? I'm running one other guest, which is a front-end QMT host that belongs to my QMT Cluster - basically the QMT ISP Array setup that Jake documented in his videos. So this front-end host is mounting the mailstore and QMT files over an NFS share, and then running Dovecot, Roundcube, and Squirrelmail. At the moment there are only 3 domains on the Cluster, and I'm still in the process of testing things. The long and the short of it, is...the only real load on the host which runs GW2 is the GW2 guest. I should think you could get rid of GW3 eventually. Yeah, that will probably happen in the not-so-distant future. The only reason I've kept it up, is for redundancy and since it is at a geographically different location than the other two GW's. What are the specs on the SA boxes? SA1 - Dell PowerEdge 2650: Dual Xeon 3.4GHz 64bit processors, 4GB RAM, 1x 73GB hdd (I need to add another and setup a RAID1) SA2 - Dell E-521: AMD Athlon 64 X2 Dual Core 3800+ processor, 4GB RAM, 1x 80GB hdd (I'd like to add another and mirror this one as well) The challenge as I see it will be getting from where you're at to where you want to be with little to no disruption. Do you have domains spread across all 3 GWs presently, or is there some redundancy? Likewise for the SA boxes? GW1-3 are all configured as closely as possible. They contain all the same domains. The main differences are that GW1 is setup to pass all mail to SA1 using smtproutes, while GW2 3 are passing mail to SA2. It might be simpler to drop off a gateway entirely and put an SA box on the edge, rather than trying to put SA functionality into a GW. Especially if you're going to end up with things on the present SA hosts anyhow. Do you have anything else virtual besides GW1? The only other things I've virtualized are my virtualmin webserver, and a couple of XMX servers which are legacy boxes from when I took over the company, and are simply CentOS installs with Sendmail configured for high volume outbound mail.
[qmailtoaster] Re: Sanesecurity, spamassassin spamdyke
On 01/24/2012 11:15 PM, Casey Price wrote: No worries Eric...I appreciate the insight! We have a few hundred domains with several thousand users. You should be able to get by with a single host in that case. Might need to beef it up a little though depending on what it's got. I meant to comment on your spamdyke config too. I'd really try to keep using the reject-unresolvable-rdns option, as it does catch a lot of spam. I've found very few legit senders that don't have this right. Typically it only happens when a server's IP address is changed and the admin overlooks this aspect. If you really need an interim fix (while the sending admin fixes their config), you can simply whitelist the domains that have a problem. This is better than disabling the filter entirely. Your scanning load will likely be reduced as a result. -- -Eric 'shubes' - Qmailtoaster is sponsored by Vickers Consulting Group (www.vickersconsulting.com) Vickers Consulting Group offers Qmailtoaster support and installations. If you need professional help with your setup, contact them today! - Please visit qmailtoaster.com for the latest news, updates, and packages. To unsubscribe, e-mail: qmailtoaster-list-unsubscr...@qmailtoaster.com For additional commands, e-mail: qmailtoaster-list-h...@qmailtoaster.com
Re: [qmailtoaster] Re: Sanesecurity, spamassassin spamdyke
On 1/25/12 10:40 AM, Eric Shubert wrote: On 01/24/2012 11:15 PM, Casey Price wrote: No worries Eric...I appreciate the insight! We have a few hundred domains with several thousand users. You should be able to get by with a single host in that case. Might need to beef it up a little though depending on what it's got. I believe our SA boxes are a bit beefier than the GW boxes, so I might just rebuild one of the SA boxes over the weekend and turn it into the new GW1 box and run spamdyke as well as spamassassin on it. My real need here is to consolidate a few of these front-end hosts. I meant to comment on your spamdyke config too. I'd really try to keep using the reject-unresolvable-rdns option, as it does catch a lot of spam. I've found very few legit senders that don't have this right. Typically it only happens when a server's IP address is changed and the admin overlooks this aspect. If you really need an interim fix (while the sending admin fixes their config), you can simply whitelist the domains that have a problem. This is better than disabling the filter entirely. Your scanning load will likely be reduced as a result. I definitely agree with you on that one Eric...I remember a few months ago when I turned that option on, and would check my spamdyke-stats script...it blocked literally like 90% of the mail. I started getting too many complaints about emails not being received, or senders getting errors when attempting to send mail to my customers. You are probably right though, just doing the whitelisting would probably remedy the issue. At the time I was trying to keep everyone happy and already had several customers that were giving me grief. I will have to look into it again though, because that would drastically reduce the load. Casey Price Smile Global Technical Support Submit or check trouble tickets http://billing.smileglobal.com www.smileglobal.com http://www.smileglobal.com Follow us on Twitter https://twitter.com/#%21/SmileInternet Find us on Facebook https://www.facebook.com/smileglobal
Re: [qmailtoaster] Re: Sanesecurity, spamassassin spamdyke
Casey Price Smile Global Technical Support Submit or check trouble tickets http://billing.smileglobal.com www.smileglobal.com http://www.smileglobal.com Follow us on Twitter https://twitter.com/#%21/SmileInternet Find us on Facebook https://www.facebook.com/smileglobal On 1/25/12 8:14 PM, Casey Price wrote: On 1/25/12 10:40 AM, Eric Shubert wrote: On 01/24/2012 11:15 PM, Casey Price wrote: No worries Eric...I appreciate the insight! We have a few hundred domains with several thousand users. You should be able to get by with a single host in that case. Might need to beef it up a little though depending on what it's got. I believe our SA boxes are a bit beefier than the GW boxes, so I might just rebuild one of the SA boxes over the weekend and turn it into the new GW1 box and run spamdyke as well as spamassassin on it. My real need here is to consolidate a few of these front-end hosts. On another note...that link that Eric previously shared from Bill Schupp's site shows spamd running on a separate host with the spamc client running on the inbound boxes. How might one go about setting up something like this, and is it recommended? I believe the reason we had separated out the GW boxes from the SA boxes was because there were times that the GW boxes would get overloaded trying to process messages using spamassassin and we'd end up with a huge queue. So if I'm interpreting this correctly, if we made the SA1 box purely a spamassassin box (which it pretty much is now, but all the mail is being passed from GW1 via smtproutes) and then had spamc running on GW1, that would probably solved some of my problems don't you think? At least the ones I had been having from SaneSecurity and it sending bounces back to my GW box. I meant to comment on your spamdyke config too. I'd really try to keep using the reject-unresolvable-rdns option, as it does catch a lot of spam. I've found very few legit senders that don't have this right. Typically it only happens when a server's IP address is changed and the admin overlooks this aspect. If you really need an interim fix (while the sending admin fixes their config), you can simply whitelist the domains that have a problem. This is better than disabling the filter entirely. Your scanning load will likely be reduced as a result. I definitely agree with you on that one Eric...I remember a few months ago when I turned that option on, and would check my spamdyke-stats script...it blocked literally like 90% of the mail. I started getting too many complaints about emails not being received, or senders getting errors when attempting to send mail to my customers. You are probably right though, just doing the whitelisting would probably remedy the issue. At the time I was trying to keep everyone happy and already had several customers that were giving me grief. I will have to look into it again though, because that would drastically reduce the load. Casey Price Smile Global Technical Support Submit or check trouble tickets http://billing.smileglobal.com www.smileglobal.com http://www.smileglobal.com Follow us on Twitter https://twitter.com/#%21/SmileInternet Find us on Facebook https://www.facebook.com/smileglobal
[qmailtoaster] Re: Sanesecurity, spamassassin spamdyke
The stock QMT configuration scans the message while the perimeter smtp session is still active, which allows it to simply reject the message (not accepting it), because it's coming directly from the sender's server. In this case, the sender's server is responsible for creating a bounce message to the sender. I don't know why GW1 is bouncing the message to the postmaster@gw1 instead of the original sender, but perhaps it tried and cannot. The way you have things set up, the SA1 host needs to go ahead and accept the message from GW1, and then generate a bounce to the original sender. This is not a very good way of handling things, as it contributes to backscatter (bounces with forged return addresses). That being said, I think there may be a way to configure qmail and simscan such that a message gets bounced (returned to sender) instead of refused (leaving the sending server (GW1) to deal with it), but I don't know about how to do, and would recommend against this configuration. When the message is denied at the perimeter, there is no bounced message (from you), and a good chance there will be less backscatter. If you really have more traffic than a single host can deal with (which is quite a lot), then there's probably a better way to distribute the load. I would let the scanning be done on (or from) the gateway server, which handles the smtp sessions, and find another way to divvy up the load if required. Sorry I can't be of more help than this. If you gave us some idea of how many domains and accounts and messages you're talking about, we might get some better idea. -- -Eric 'shubes' On 01/24/2012 06:43 PM, Casey Price wrote: Any takers on this one? The problem is definitely on my SA1 box (you can see spamd start hogging memory and eating up the processor and notice a constant heavy load when you view the stats with htop, or w. There isn't really much on the wiki regarding SaneSecurity, so I was hoping for some insight in configuring it and tuning it for better performance. So, while this is one piece to the problem, the other issue is that when messages are flagged by SaneSecurity, they are rejected by SA1 (primary spamassassin box) when GW1 (primary spamdyke box - all mail hits this server, then is passed to SA1 using smtproutes) attempts to pass the mail to the next hop. What this means is that I end up with several thousand messages in my queue every day on GW1, and they end up being something like this: 15107007 (9, L) Return-path: #@[] From: mailer-dae...@gateway1.smileglobal.com To: postmas...@gateway1.smileglobal.com Subject: failure notice Date: 25 Jan 2012 00:50:42 - Size: 23018 bytes -- Hi. This is the qmail-send program at gateway1.smileglobal.com. I'm afraid I wasn't able to deliver your message to the following addresses. This is a permanent error; I've given up. Sorry it didn't work out. r...@some-domain.com: User and password not set, continuing without authentication. r...@some-domain.com 69.7.35.24 failed after I sent the message. Remote host said: 554 Your email was rejected because it contains the Sanesecurity.Jurlbl.5049.UNOFFICIAL virus Hoping someone can shed some light on this for me and help me figure out a better solution. Thanks, Casey Price Smile Global Technical Support Submit or check trouble tickets http://billing.smileglobal.com www.smileglobal.com http://www.smileglobal.com Follow us on Twitter https://twitter.com/#%21/SmileInternet Find us on Facebook https://www.facebook.com/smileglobal On 1/19/12 6:12 PM, Casey Price wrote: Hi guys, Lately I've been noticing the queue on one of my gateway servers (running QMT with spamdyke) has been growing quite large on a daily basis. Once mail hits this server it is passed on to my SA box which also runs QMT with clamav spamassassin. I recently used the qtp-install-sanesecurity script, and while it appears to be properly identifying mail, it ends up rejecting the mail as it is being passed on from the gateway server. So it ends up back in the gateway queue and just sits there. Is there a way I can prevent the SA box from rejecting and sending the mail back to the gateway box? It would be nice if it just deleted the mail. I'm using simscan on the SA box as well. Any recommendations? I previously had the following options enabled in spamdyke, but ended up turning them off because many of my customers were complaining about not receiving their mail... reject-ip-in-cc-rdns reject-unresolvable-rdns Thanks -- Casey Price Smile Global Technical Support Submit or check trouble tickets http://billing.smileglobal.com www.smileglobal.com http://www.smileglobal.com Follow us on Twitter https://twitter.com/#%21/SmileInternet Find us on Facebook https://www.facebook.com/smileglobal - Qmailtoaster is sponsored by Vickers Consulting
[qmailtoaster] Re: Sanesecurity, spamassassin spamdyke
Here's a setup that I like: http://www.shupp.org/maps/ispcluster.html Notice that spamd scanning is offloaded, but it's done while the smtp (mx) session stays open, so that messages can be rejected, not bounced. I'm glad this link is still up. I just found out that Bill appears to have taken most of his Qmail stuff down. Would someone care to get Bill's permission first, then put this up on the wiki? I think this is worthy. -- -Eric 'shubes' On 01/24/2012 06:43 PM, Casey Price wrote: Any takers on this one? The problem is definitely on my SA1 box (you can see spamd start hogging memory and eating up the processor and notice a constant heavy load when you view the stats with htop, or w. There isn't really much on the wiki regarding SaneSecurity, so I was hoping for some insight in configuring it and tuning it for better performance. So, while this is one piece to the problem, the other issue is that when messages are flagged by SaneSecurity, they are rejected by SA1 (primary spamassassin box) when GW1 (primary spamdyke box - all mail hits this server, then is passed to SA1 using smtproutes) attempts to pass the mail to the next hop. What this means is that I end up with several thousand messages in my queue every day on GW1, and they end up being something like this: 15107007 (9, L) Return-path: #@[] From: mailer-dae...@gateway1.smileglobal.com To: postmas...@gateway1.smileglobal.com Subject: failure notice Date: 25 Jan 2012 00:50:42 - Size: 23018 bytes -- Hi. This is the qmail-send program at gateway1.smileglobal.com. I'm afraid I wasn't able to deliver your message to the following addresses. This is a permanent error; I've given up. Sorry it didn't work out. r...@some-domain.com: User and password not set, continuing without authentication. r...@some-domain.com 69.7.35.24 failed after I sent the message. Remote host said: 554 Your email was rejected because it contains the Sanesecurity.Jurlbl.5049.UNOFFICIAL virus Hoping someone can shed some light on this for me and help me figure out a better solution. Thanks, Casey Price Smile Global Technical Support Submit or check trouble tickets http://billing.smileglobal.com www.smileglobal.com http://www.smileglobal.com Follow us on Twitter https://twitter.com/#%21/SmileInternet Find us on Facebook https://www.facebook.com/smileglobal On 1/19/12 6:12 PM, Casey Price wrote: Hi guys, Lately I've been noticing the queue on one of my gateway servers (running QMT with spamdyke) has been growing quite large on a daily basis. Once mail hits this server it is passed on to my SA box which also runs QMT with clamav spamassassin. I recently used the qtp-install-sanesecurity script, and while it appears to be properly identifying mail, it ends up rejecting the mail as it is being passed on from the gateway server. So it ends up back in the gateway queue and just sits there. Is there a way I can prevent the SA box from rejecting and sending the mail back to the gateway box? It would be nice if it just deleted the mail. I'm using simscan on the SA box as well. Any recommendations? I previously had the following options enabled in spamdyke, but ended up turning them off because many of my customers were complaining about not receiving their mail... reject-ip-in-cc-rdns reject-unresolvable-rdns Thanks -- Casey Price Smile Global Technical Support Submit or check trouble tickets http://billing.smileglobal.com www.smileglobal.com http://www.smileglobal.com Follow us on Twitter https://twitter.com/#%21/SmileInternet Find us on Facebook https://www.facebook.com/smileglobal - Qmailtoaster is sponsored by Vickers Consulting Group (www.vickersconsulting.com) Vickers Consulting Group offers Qmailtoaster support and installations. If you need professional help with your setup, contact them today! - Please visit qmailtoaster.com for the latest news, updates, and packages. To unsubscribe, e-mail: qmailtoaster-list-unsubscr...@qmailtoaster.com For additional commands, e-mail: qmailtoaster-list-h...@qmailtoaster.com
Re: [qmailtoaster] Re: Sanesecurity, spamassassin spamdyke
On 1/24/12 6:43 PM, Eric Shubert wrote: The stock QMT configuration scans the message while the perimeter smtp session is still active, which allows it to simply reject the message (not accepting it), because it's coming directly from the sender's server. In this case, the sender's server is responsible for creating a bounce message to the sender. I don't know why GW1 is bouncing the message to the postmaster@gw1 instead of the original sender, but perhaps it tried and cannot. The way you have things set up, the SA1 host needs to go ahead and accept the message from GW1, and then generate a bounce to the original sender. This is not a very good way of handling things, as it contributes to backscatter (bounces with forged return addresses). That being said, I think there may be a way to configure qmail and simscan such that a message gets bounced (returned to sender) instead of refused (leaving the sending server (GW1) to deal with it), but I don't know about how to do, and would recommend against this configuration. I agree with you on this one...I don't really like the way things are setup up at the moment. This is how things were setup when I took over, so I'm thinking I'd like to do away with my SA1 SA2 boxes and just beef up the two GW boxes and run spamassassin on them. Right now it is inefficient, because the bounces end up back in the GW queues and just waste resources. When the message is denied at the perimeter, there is no bounced message (from you), and a good chance there will be less backscatter. If you really have more traffic than a single host can deal with (which is quite a lot), then there's probably a better way to distribute the load. I would let the scanning be done on (or from) the gateway server, which handles the smtp sessions, and find another way to divvy up the load if required. Yeah, the method you are suggesting makes much more sense and seems like it would be much more effective and less-resource intensive overall compared to our current config. Sorry I can't be of more help than this. If you gave us some idea of how many domains and accounts and messages you're talking about, we might get some better idea. No worries Eric...I appreciate the insight! We have a few hundred domains with several thousand users. Oh and thanks for sending that link...I've been to that page before, but not in awhile. Anyone come across good documentation on setting up spamd on a separate machine and then using the spamc client? Casey Price Smile Global Technical Support Submit or check trouble tickets http://billing.smileglobal.com www.smileglobal.com http://www.smileglobal.com Follow us on Twitter https://twitter.com/#%21/SmileInternet Find us on Facebook https://www.facebook.com/smileglobal
